Reinforcement learning differs from supervised learning in not needing labelled input/output pairs be presented, and in not needing sub-optimal actions to be explicitly corrected. Instead the focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).

The environment is typically stated in the form of a Markov decision process (MDP), because many reinforcement learning algorithms for this context use dynamic programming techniques. The main difference between the classical dynamic programming methods and reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the MDP and they target large MDPs where exact methods become infeasible.

Due to its generality, reinforcement learning is studied in many disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, and statistics. In the operations research and control literature, reinforcement learning is called approximate dynamic programming, or neuro-dynamic programming. The problems of interest in reinforcement learning have also been studied in the theory of optimal control, which is concerned mostly with the existence and characterization of optimal solutions, and algorithms for their exact computation, and less with learning or approximation, particularly in the absence of a mathematical model of the environment. In economics and game theory, reinforcement learning may be used to explain how equilibrium may arise under bounded rationality.

Basic reinforcement is modeled as a Markov decision process (MDP). In mathematics, a Markov decision process (MDP) is a discrete-time stochastic control process. It provides a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying optimization problems solved via dynamic programming. MDPs were known at least as early as the 1950s. A core body of research on Markov decision processes resulted from Ronald Howard’s 1960 book, Dynamic Programming and Markov Processes. They are used in many disciplines, including robotics, automatic control, economics and manufacturing. The name of MDPs comes from the Russian mathematician Andrey Markov as they are an extension of Markov chains.

At each time step, the process is in some state S, and the decision maker may choose any action a that is available in state S. The process responds at the next time step by randomly moving into a new state S’, and giving the decision maker a corresponding reward Ra(S,S’).

The probability that the process moves into its new state S’ is influenced by the chosen action a. Specifically, it is given by the state transition function Pa(S,S’). Thus, the next state S’ depends on the current state S and the decision maker’s action a. But given S and a, it is conditionally independent of all previous states and actions. In other words, the state transitions of an MDP satisfy the Markov property.

Markov decision processes are an extension of Markov chains; the difference is the addition of actions (allowing choice) and rewards (giving motivation). Conversely, if only one action exists for each state (e.g. “wait”) and all rewards are the same (e.g. “zero”), a Markov decision process reduces to a Markov chain.

A reinforcement learning agent interacts with its environment in discrete time steps. At each time t, the agent receives the current state S(t) and reward r(t). It then chooses an action a(t) from the set of available actions, which is subsequently sent to the environment. The environment moves to a new state S(t+1) and the reward r(t+1) associated with the transition is determined. The goal of a reinforcement learning agent is to learn a policy which maximizes the expected cumulative reward.

Formulating the problem as a MDP assumes the agent directly observes the current environmental state. In this case the problem is said to have full observability. If the agent only has access to a subset of states, or if the observed states are corrupted by noise, the agent is said to have partial observability, and formally the problem must be formulated as a Partially observable Markov decision process. In both cases, the set of actions available to the agent can be restricted. For example, the state of an account balance could be restricted to be positive; if the current value of the state is 3 and the state transition attempts to reduce the value by 4, the transition will not be allowed.

When the agent’s performance is compared to that of an agent that acts optimally, the difference in performance gives rise to the notion of regret. In order to act near optimally, the agent must reason about the long-term consequences of its actions (i.e., maximize future income), although the immediate reward associated with this might be negative.

Thus, reinforcement learning is particularly well-suited to problems that include a long-term versus short-term reward trade-off. It has been applied successfully to various problems, including robot control, elevator scheduling, telecommunications, backgammon, checkers and Go (AlphaGo).

Two elements make reinforcement learning powerful: the use of samples to optimize performance and the use of function approximation to deal with large environments. Thanks to these two key components, reinforcement learning can be used in large environments in the following situations:

• A model of the environment is known, but an analytic solution is not available.
• Only a simulation model of the environment is given (the subject of simulation-based optimization).
• The only way to collect information about the environment is to interact with it.

The first two of these problems could be considered planning problems (since some form of model is available), while the last one could be considered to be a genuine learning problem. However, reinforcement learning converts both planning problems to machine learning problems.

The exploration vs. exploitation trade-off has been most thoroughly studied through the multi-armed bandit problem and for finite state space MDPs in Burnetas and Katehakis (1997).

Reinforcement learning requires clever exploration mechanisms; randomly selecting actions, without reference to an estimated probability distribution, shows poor performance. The case of (small) finite Markov decision processes is relatively well understood. However, due to the lack of algorithms that scale well with the number of states (or scale to problems with infinite state spaces), simple exploration methods are the most practical.

Even if the issue of exploration is disregarded and even if the state was observable, the problem remains to use past experience to find out which actions lead to higher cumulative rewards.

Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. The adjective “deep” in deep learning refers to the use of multiple layers in the network. Early work showed that a linear perceptron cannot be a universal classifier, and then that a network with a nonpolynomial activation function with one hidden layer of unbounded width can on the other hand so be. Deep learning is a modern variation which is concerned with an unbounded number of layers of bounded size, which permits practical application and optimized implementation, while retaining theoretical universality under mild conditions. In deep learning the layers are also permitted to be heterogeneous and to deviate widely from biologically informed connectionist models, for the sake of efficiency, trainability and understandability, whence the “structured” part.

TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache License 2.0 in 2015.

Starting in 2011, Google Brain built DistBelief as a proprietary machine learning system based on deep learning neural networks. Its use grew rapidly across diverse Alphabet companies in both research and commercial applications. Google assigned multiple computer scientists, including Jeff Dean, to simplify and refactor the codebase of DistBelief into a faster, more robust application-grade library, which became TensorFlow. In 2009, the team, led by Geoffrey Hinton, had implemented generalized backpropagation and other improvements which allowed generation of neural networks with substantially higher accuracy, for instance a 25% reduction in errors in speech recognition.

TensorFlow is Google Brain’s second-generation system. Version 1.0.0 was released on February 11, 2017. While the reference implementation runs on single devices, TensorFlow can run on multiple CPUs and GPUs (with optional CUDA and SYCL extensions for general-purpose computing on graphics processing units). TensorFlow is available on 64-bit Linux, macOS, Windows, and mobile computing platforms including Android and iOS. Its flexible architecture allows for the easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. TensorFlow computations are expressed as stateful dataflow graphs. The name TensorFlow derives from the operations that such neural networks perform on multidimensional data arrays, which are referred to as tensors. During the Google I/O Conference in June 2016, Jeff Dean stated that 1,500 repositories on GitHub mentioned TensorFlow, of which only 5 were from Google. In December 2017, developers from Google, Cisco, RedHat, CoreOS, and CaiCloud introduced Kubeflow at a conference. Kubeflow allows operation and deployment of TensorFlow on Kubernetes. In March 2018, Google announced TensorFlow.js version 1.0 for machine learning in JavaScript. In Jan 2019, Google announced TensorFlow 2.0. It became officially available in Sep 2019. In May 2019, Google announced TensorFlow Graphics for deep learning in computer graphics.

Quantum computing began in the early 1980s, when physicist Paul Benioff proposed a quantum mechanical model of the Turing machine. Richard Feynman and Yuri Manin later suggested that a quantum computer had the potential to simulate things that a classical computer could not. In 1994, Peter Shor developed a quantum algorithm for factoring integers that had the potential to decrypt RSA-encrypted communications. Despite ongoing experimental progress since the late 1990s, most researchers believe that “fault-tolerant quantum computing is still a rather distant dream.”. In recent years, investment into quantum computing research has increased in both the public and private sector. On 23 October 2019, Google AI, in partnership with the U.S. National Aeronautics and Space Administration (NASA), claimed to have performed a quantum computation that is infeasible on any classical computer (so-called quantum supremacy result).

There are several models of quantum computers (or rather, quantum computing systems), including the quantum circuit model, quantum Turing machine, adiabatic quantum computer, one-way quantum computer, and various quantum cellular automata. The most widely used model is the quantum circuit. Quantum circuits are based on the quantum bit, or “qubit”, which is somewhat analogous to the bit in classical computation. Qubits can be in a 1 or 0 quantum state, or they can be in a superposition of the 1 and 0 states. However, when qubits are measured the result of the measurement is always either a 0 or a 1; the probabilities of these two outcomes depend on the quantum state that the qubits were in immediately prior to the measurement.

Progress towards building a physical quantum computer focuses on technologies such as transmons, ion traps and topological quantum computers, which aim to create high-quality qubits. These qubits may be designed differently, depending on the full quantum computer’s computing model, whether quantum logic gates, quantum annealing, or adiabatic quantum computation. There are currently a number of significant obstacles in the way of constructing useful quantum computers. In particular, it is difficult to maintain the quantum states of qubits as they suffer from quantum decoherence and state fidelity. Quantum computers therefore require error correction. Any computational problem that can be solved by a classical computer can also be solved by a quantum computer. Conversely, any problem that can be solved by a quantum computer can also be solved by a classical computer, at least in principle given enough time. In other words, quantum computers obey the Church–Turing thesis. While this means that quantum computers provide no additional advantages over classical computers in terms of computability, quantum algorithms for certain problems have significantly lower time complexities than corresponding known classical algorithms. Notably, quantum computers are believed to be able to quickly solve certain problems that no classical computer could solve in any feasible amount of time—a feat known as “quantum supremacy.” The study of the computational complexity of problems with respect to quantum computers is known as quantum complexity theory.

In 2019, Sycamore completed a task in 200 seconds that Google claimed, in a Nature paper, would take a state-of-the-art supercomputer 10,000 years to finish. Thus, Google claimed to have achieved quantum supremacy. To estimate the time that would be taken by a classical supercomputer, Google ran portions of the quantum circuit simulation on the Summit, the most powerful classical computer in the world. Later, IBM made a counter-argument, claiming that the task would only take 2.5 days on a classical system like Summit. If Google’s claims are upheld, then it would represent an exponential leap in computing power.

In August 2020 quantum engineers working for Google reported the largest chemical simulation on a quantum computer – a Hartree-Fock approximation with Sycamore paired with a classical computer that analyzed results to provide new parameters for the 12-qubit system.

In December 2020, the Chinese photon-based Jiuzhang processor, developed by USTC, achieved a processing power of 76 qubits and was 10 billion times faster than Sycamore, making it the second computer to attain quantum supremacy.

The Quantum AI Lab was announced by Google Research in a blog post on May 16, 2013. At the time of launch, the Lab was using the most advanced commercially available quantum computer, D-Wave Two from D-Wave Systems.

On May 20, 2013, it was announced that people could apply to use time on the D-Wave Two at the Lab. On October 10, 2013, Google released a short film describing the current state of the Quantum AI Lab. On October 18, 2013, Google announced that it had incorporated quantum physics into Minecraft.

In January 2014, Google reported results comparing the performance of the D-Wave Two in the lab with that of classical computers. The results were ambiguous and provoked heated discussion on the Internet. On 2 September 2014, it was announced that the Quantum AI Lab, in partnership with UC Santa Barbara, would be launching an initiative to create quantum information processors based on superconducting electronics.

On the 23rd of October 2019, the Quantum AI Lab announced in a paper that it had achieved quantum supremacy.

Quantum computing is considered to help in development of the innovations of tomorrow, including AI. That’s why Google committs significant resources to building dedicated quantum hardware and software.

Quantum computing is a new paradigm that will play a big role in accelerating tasks for AI. Google aims to offer researchers and developers access to open source frameworks and computing power that can operate beyond classical capabilities of computation.

The main focus areas of Google AI Quantum are

• Superconducting qubit processors: Superconducting qubits with chip-based scalable architecture targeting two-qubit gate error < 0.5%.
• Qubit metrology: Reducing two-qubit loss below 0.2% is critical for error correction. We are working on a quantum supremacy experiment, to approximately sample a quantum circuit beyond the capabilities of state-of-the-art classical computers and algorithms.
• Quantum simulation: Simulation of physical systems is among the most anticipated applications of quantum computing. We especially focus on quantum algorithms for modelling systems of interacting electrons with applications in chemistry and materials science.
• Quantum assisted optimization: We are developing hybrid quantum-classical solvers for approximate optimization. Thermal jumps in classical algorithms to overcome energy barriers could be enhanced by invoking quantum updates. We are in particular interested in coherent population transfer.
• Quantum neural networks: We are developing a framework to implement a quantum neural network on near-term processors. We are interested in understanding what advantages may arise from generating massive superposition states during operation of the network.

The main tools developed by Google AI Quantum are open-source frameworks specifically designed for developing novel quantum algorithms to help solve near-term applications for practical problems. These include:

• Cirq: an open-source quantum framework for building and experimenting with noisy intermediate scale quantum (NISQ) algorithms on near-term quantum processors
• OpenFermion: an open-source platform for translating problems in chemistry and materials science into quantum circuits that can be executed on existing platforms

Google AI Quantum near-term applications include:

Quantum Simulation

The design of new materials and elucidation of complex physics through accurate simulations of chemistry and condensed matter models are among the most promising applications of quantum computing.

Error mitigation techniques

We work to develop methods on the road to full quantum error correction that have the capability of dramatically reducing noise in current devices. While full-scale fault tolerant quantum computing may require considerable developments, we have developed the quantum subspace expansion technique to help utilize techniques from quantum error correction to improve performance of applications on near-term devices. Moreover, these techniques facilitate testing of complex quantum codes on near-term devices. We are actively pushing these techniques into new areas and leveraging them as a basis for design of near term experiments.

Quantum Machine Learning

We are developing hybrid quantum-classical machine learning techniques on near-term quantum devices. We are studying universal quantum circuit learning for classification and clustering of quantum and classical data. We are also interested in generative and discriminative quantum neural networks, that could be used as quantum repeaters and state purification units within quantum communication networks, or for verification of other quantum circuits.

Quantum Optimization

Discrete optimizations in aerospace, automotive, and other industries may benefit from hybrid quantum-classical optimization, for example simulated annealing, quantum assisted optimization algorithm (QAOA) and quantum enhanced population transfer may have utility with today’s processors.

Python is dynamically-typed (executing at runtime many common programming behaviours that static programming languages perform during compilation) and garbage-collected (with automatic memory management). It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented and functional programming. It was created in the late 1980s, and first released in 1991, by Guido van Rossum as a successor to the ABC programming language. Python 2.0, released in 2000, introduced new features, such as list comprehensions, and a garbage collection system with reference counting, and was discontinued with version 2.7 in 2020. Python 3.0, released in 2008, was a major revision of the language that is not completely backward-compatible and much Python 2 code does not run unmodified on Python 3. With Python 2’s end-of-life (and pip having dropped support in 2021), only Python 3.6.x and later are supported, with older versions still supporting e.g. Windows 7 (and old installers not restricted to 64-bit Windows).

Python interpreters are supported for mainstream operating systems and available for a few more (and in the past supported many more). A global community of programmers develops and maintains CPython, a free and open-source reference implementation. A non-profit organization, the Python Software Foundation, manages and directs resources for Python and CPython development.

As of January 2021, Python ranks third in TIOBE’s index of most popular programming languages, behind C and Java, having previously gained second place and their award for the most popularity gain for 2020. It was selected Programming Language of the Year in 2007, 2010, and 2018.

An empirical study found that scripting languages, such as Python, are more productive than conventional languages, such as C and Java, for programming problems involving string manipulation and search in a dictionary, and determined that memory consumption was often “better than Java and not much worse than C or C++”. Large organizations that use Python include i.a. Wikipedia, Google, Yahoo!, CERN, NASA, Facebook, Amazon, Instagram.

Beyond its artificial intelligence applications, Python, as a scripting language with modular architecture, simple syntax and rich text processing tools, is often used for natural language processing.

Google Cloud offers two computer vision services (jointly referred to as Vision AI) that use machine learning to understand images and videos with high prediction accuracy, i.e. AutoML Vision and the Vision API. The AutoML Vision automates training of the custom machine learning models. It enables upload of images and training custom image models with easy-to-use graphical interface; optimizing the models for accuracy, latency, and size and exporting them to any application in the cloud, or to an array of devices at the edge. On the other hand Google Cloud’s Vision API offers powerful pre-trained machine learning models through REST (Representational State Transfer) and RPC (Remote Procedure Call) APIs, assigning labels to images and quickly classifying them into millions of predefined categories. Detecting objects and faces, reading printed and handwritten text, and building valuable metadata into image catalogs. You can thus use AutoML Vision to derive insights from images in the cloud or at the edge or use the pre-trained Vision API models to detect emotion, understand text from visual data, and more.

With Google Cloud’s Vision API it is possible to:

• Detect objects: Detect objects, where they are, and how many.
• Enable vision product search: Compare photos to images in your product catalog, and return a ranked list of similar items.
• Detect printed and handwritten text: Use OCR and automatically identify language.
• Detect faces: Detect faces and facial attributes. (Face recognition not supported.)
• Identify popular places and product logos: Automatically identify well-known landmarks and product logos.
• Assign general image attributes: Detect general attributes and appropriate crop hints.
• Detect web entities and pages: Find news events, logos, and similar images on the web.
• Moderate content: Detect explicit content (adult, violent, etc.) within images.
• Celebrity recognition: Identify celebrity faces in images (limited access, see documentation.)
• Classify images using predefined labels: Pre-trained models leverage vast libraries of predefined labels.
• Use Google’s data labeling service: Google can help annotating images, videos, and text.
• Use APIs: Use REST and RPC APIs.

The possible use cases for Vision API are countless.

For example using Vision API you can implement vision product search enabling your customers to find products of interest within images and visually search product catalogs (image search, automatically designated similar products, etc.).

The video above explains how Google’s Cloud AutoML Vision uses AI to analyze images.

A twin AI system, closely related to the pre-trained and constantly upgraded Google Vision API is Google AutoML Vision enabling enterprises to use their own machine learning models and custom training for the artificial intelligence assistance in vision analysis and understanding. Part of Google Cloud’s machine learning suite of products, it’s designed to help developers with limited machine learning expertise train custom vision models for their specific use cases. In need of on-demand access to the general pre-trained model, AI developers hould use the Google Vision API.

Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. The adjective “deep” in deep learning refers to the use of multiple layers in the network. Early work showed that a linear perceptron cannot be a universal classifier, and then that a network with a nonpolynomial activation function with one hidden layer of unbounded width can on the other hand so be. Deep learning is a modern variation which is concerned with an unbounded number of layers of bounded size, which permits practical application and optimized implementation, while retaining theoretical universality under mild conditions. In deep learning the layers are also permitted to be heterogeneous and to deviate widely from biologically informed connectionist models, for the sake of efficiency, trainability and understandability, whence the “structured” part.

Python is dynamically-typed (executing at runtime many common programming behaviours that static programming languages perform during compilation) and garbage-collected (with automatic memory management). It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented and functional programming. It was created in the late 1980s, and first released in 1991, by Guido van Rossum as a successor to the ABC programming language. Python 2.0, released in 2000, introduced new features, such as list comprehensions, and a garbage collection system with reference counting, and was discontinued with version 2.7 in 2020. Python 3.0, released in 2008, was a major revision of the language that is not completely backward-compatible and much Python 2 code does not run unmodified on Python 3. With Python 2’s end-of-life (and pip having dropped support in 2021), only Python 3.6.x and later are supported, with older versions still supporting e.g. Windows 7 (and old installers not restricted to 64-bit Windows).

Python interpreters are supported for mainstream operating systems and available for a few more (and in the past supported many more). A global community of programmers develops and maintains CPython, a free and open-source reference implementation. A non-profit organization, the Python Software Foundation, manages and directs resources for Python and CPython development.

As of January 2021, Python ranks third in TIOBE’s index of most popular programming languages, behind C and Java, having previously gained second place and their award for the most popularity gain for 2020. It was selected Programming Language of the Year in 2007, 2010, and 2018.

An empirical study found that scripting languages, such as Python, are more productive than conventional languages, such as C and Java, for programming problems involving string manipulation and search in a dictionary, and determined that memory consumption was often “better than Java and not much worse than C or C++”. Large organizations that use Python include i.a. Wikipedia, Google, Yahoo!, CERN, NASA, Facebook, Amazon, Instagram.

Beyond its artificial intelligence applications, Python, as a scripting language with modular architecture, simple syntax and rich text processing tools, is often used for natural language processing.

TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache License 2.0 in 2015.

Starting in 2011, Google Brain built DistBelief as a proprietary machine learning system based on deep learning neural networks. Its use grew rapidly across diverse Alphabet companies in both research and commercial applications. Google assigned multiple computer scientists, including Jeff Dean, to simplify and refactor the codebase of DistBelief into a faster, more robust application-grade library, which became TensorFlow. In 2009, the team, led by Geoffrey Hinton, had implemented generalized backpropagation and other improvements which allowed generation of neural networks with substantially higher accuracy, for instance a 25% reduction in errors in speech recognition.

TensorFlow is Google Brain’s second-generation system. Version 1.0.0 was released on February 11, 2017. While the reference implementation runs on single devices, TensorFlow can run on multiple CPUs and GPUs (with optional CUDA and SYCL extensions for general-purpose computing on graphics processing units). TensorFlow is available on 64-bit Linux, macOS, Windows, and mobile computing platforms including Android and iOS. Its flexible architecture allows for the easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. TensorFlow computations are expressed as stateful dataflow graphs. The name TensorFlow derives from the operations that such neural networks perform on multidimensional data arrays, which are referred to as tensors. During the Google I/O Conference in June 2016, Jeff Dean stated that 1,500 repositories on GitHub mentioned TensorFlow, of which only 5 were from Google. In December 2017, developers from Google, Cisco, RedHat, CoreOS, and CaiCloud introduced Kubeflow at a conference. Kubeflow allows operation and deployment of TensorFlow on Kubernetes. In March 2018, Google announced TensorFlow.js version 1.0 for machine learning in JavaScript. In Jan 2019, Google announced TensorFlow 2.0. It became officially available in Sep 2019. In May 2019, Google announced TensorFlow Graphics for deep learning in computer graphics.

Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. The adjective “deep” in deep learning refers to the use of multiple layers in the network. Early work showed that a linear perceptron cannot be a universal classifier, and then that a network with a nonpolynomial activation function with one hidden layer of unbounded width can on the other hand so be. Deep learning is a modern variation which is concerned with an unbounded number of layers of bounded size, which permits practical application and optimized implementation, while retaining theoretical universality under mild conditions. In deep learning the layers are also permitted to be heterogeneous and to deviate widely from biologically informed connectionist models, for the sake of efficiency, trainability and understandability, whence the “structured” part.

Python is dynamically-typed (executing at runtime many common programming behaviours that static programming languages perform during compilation) and garbage-collected (with automatic memory management). It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented and functional programming. It was created in the late 1980s, and first released in 1991, by Guido van Rossum as a successor to the ABC programming language. Python 2.0, released in 2000, introduced new features, such as list comprehensions, and a garbage collection system with reference counting, and was discontinued with version 2.7 in 2020. Python 3.0, released in 2008, was a major revision of the language that is not completely backward-compatible and much Python 2 code does not run unmodified on Python 3. With Python 2’s end-of-life (and pip having dropped support in 2021), only Python 3.6.x and later are supported, with older versions still supporting e.g. Windows 7 (and old installers not restricted to 64-bit Windows).

Python interpreters are supported for mainstream operating systems and available for a few more (and in the past supported many more). A global community of programmers develops and maintains CPython, a free and open-source reference implementation. A non-profit organization, the Python Software Foundation, manages and directs resources for Python and CPython development.

As of January 2021, Python ranks third in TIOBE’s index of most popular programming languages, behind C and Java, having previously gained second place and their award for the most popularity gain for 2020. It was selected Programming Language of the Year in 2007, 2010, and 2018.

An empirical study found that scripting languages, such as Python, are more productive than conventional languages, such as C and Java, for programming problems involving string manipulation and search in a dictionary, and determined that memory consumption was often “better than Java and not much worse than C or C++”. Large organizations that use Python include i.a. Wikipedia, Google, Yahoo!, CERN, NASA, Facebook, Amazon, Instagram.

Beyond its artificial intelligence applications, Python, as a scripting language with modular architecture, simple syntax and rich text processing tools, is often used for natural language processing.

PyTorch provides two high-level features:

• Tensor computing (like NumPy) with strong acceleration via graphics processing units (GPU)
• Deep neural networks built on a tape-based automatic (computational) differentiation system

Facebook operates both PyTorch and Convolutional Architecture for Fast Feature Embedding (Caffe2), but models defined by the two frameworks were mutually incompatible. The Open Neural Network Exchange (ONNX) project was created by Facebook and Microsoft in September 2017 for converting models between frameworks. Caffe2 was merged into PyTorch at the end of March 2018.

PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on a CUDA-capable Nvidia GPU. PyTorch supports various sub-types of Tensors.

There are few important modules for Pytorch. These include:

• Autograd module: PyTorch uses a method called automatic differentiation. A recorder records what operations have performed, and then it replays it backward to compute the gradients. This method is especially powerful when building neural networks to save time on one epoch by calculating differentiation of the parameters at the forward pass.
• Optim module: torch.optim is a module that implements various optimization algorithms used for building neural networks. Most of the commonly used methods are already supported, so there is no need to build them from scratch.
• nn module: PyTorch autograd makes it easy to define computational graphs and take gradients, but raw autograd can be a bit too low-level for defining complex neural networks. This is where the nn module can help.

Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. The adjective “deep” in deep learning refers to the use of multiple layers in the network. Early work showed that a linear perceptron cannot be a universal classifier, and then that a network with a nonpolynomial activation function with one hidden layer of unbounded width can on the other hand so be. Deep learning is a modern variation which is concerned with an unbounded number of layers of bounded size, which permits practical application and optimized implementation, while retaining theoretical universality under mild conditions. In deep learning the layers are also permitted to be heterogeneous and to deviate widely from biologically informed connectionist models, for the sake of efficiency, trainability and understandability, whence the “structured” part.

Python is dynamically-typed (executing at runtime many common programming behaviours that static programming languages perform during compilation) and garbage-collected (with automatic memory management). It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented and functional programming. It was created in the late 1980s, and first released in 1991, by Guido van Rossum as a successor to the ABC programming language. Python 2.0, released in 2000, introduced new features, such as list comprehensions, and a garbage collection system with reference counting, and was discontinued with version 2.7 in 2020. Python 3.0, released in 2008, was a major revision of the language that is not completely backward-compatible and much Python 2 code does not run unmodified on Python 3. With Python 2’s end-of-life (and pip having dropped support in 2021), only Python 3.6.x and later are supported, with older versions still supporting e.g. Windows 7 (and old installers not restricted to 64-bit Windows).

Python interpreters are supported for mainstream operating systems and available for a few more (and in the past supported many more). A global community of programmers develops and maintains CPython, a free and open-source reference implementation. A non-profit organization, the Python Software Foundation, manages and directs resources for Python and CPython development.

As of January 2021, Python ranks third in TIOBE’s index of most popular programming languages, behind C and Java, having previously gained second place and their award for the most popularity gain for 2020. It was selected Programming Language of the Year in 2007, 2010, and 2018.

An empirical study found that scripting languages, such as Python, are more productive than conventional languages, such as C and Java, for programming problems involving string manipulation and search in a dictionary, and determined that memory consumption was often “better than Java and not much worse than C or C++”. Large organizations that use Python include i.a. Wikipedia, Google, Yahoo!, CERN, NASA, Facebook, Amazon, Instagram.

Beyond its artificial intelligence applications, Python, as a scripting language with modular architecture, simple syntax and rich text processing tools, is often used for natural language processing.

TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache License 2.0 in 2015.

Starting in 2011, Google Brain built DistBelief as a proprietary machine learning system based on deep learning neural networks. Its use grew rapidly across diverse Alphabet companies in both research and commercial applications. Google assigned multiple computer scientists, including Jeff Dean, to simplify and refactor the codebase of DistBelief into a faster, more robust application-grade library, which became TensorFlow. In 2009, the team, led by Geoffrey Hinton, had implemented generalized backpropagation and other improvements which allowed generation of neural networks with substantially higher accuracy, for instance a 25% reduction in errors in speech recognition.

TensorFlow is Google Brain’s second-generation system. Version 1.0.0 was released on February 11, 2017. While the reference implementation runs on single devices, TensorFlow can run on multiple CPUs and GPUs (with optional CUDA and SYCL extensions for general-purpose computing on graphics processing units). TensorFlow is available on 64-bit Linux, macOS, Windows, and mobile computing platforms including Android and iOS. Its flexible architecture allows for the easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. TensorFlow computations are expressed as stateful dataflow graphs. The name TensorFlow derives from the operations that such neural networks perform on multidimensional data arrays, which are referred to as tensors. During the Google I/O Conference in June 2016, Jeff Dean stated that 1,500 repositories on GitHub mentioned TensorFlow, of which only 5 were from Google. In December 2017, developers from Google, Cisco, RedHat, CoreOS, and CaiCloud introduced Kubeflow at a conference. Kubeflow allows operation and deployment of TensorFlow on Kubernetes. In March 2018, Google announced TensorFlow.js version 1.0 for machine learning in JavaScript. In Jan 2019, Google announced TensorFlow 2.0. It became officially available in Sep 2019. In May 2019, Google announced TensorFlow Graphics for deep learning in computer graphics.

Up until version 2.3 Keras supported multiple backends, including TensorFlow, Microsoft Cognitive Toolkit, Theano, and PlaidML. As of version 2.4, only TensorFlow is supported. Designed to enable fast experimentation with deep neural networks, it focuses on being user-friendly, modular, and extensible. It was developed as part of the research effort of project ONEIROS (Open-ended Neuro-Electronic Intelligent Robot Operating System), and its primary author and maintainer is François Chollet, a Google engineer. Chollet also is the author of the XCeption deep neural network model.

Keras contains numerous implementations of commonly used neural-network building blocks such as layers, objectives, activation functions, optimizers, and a host of tools to make working with image and text data easier to simplify the coding necessary for writing deep neural network code. The code is hosted on GitHub, and community support forums include the GitHub issues page, and a Slack channel.

In addition to standard neural networks, Keras has support for convolutional and recurrent neural networks. It supports other common utility layers like dropout, batch normalization, and pooling. Keras allows users to productize deep models on smartphones (iOS and Android), on the web, or on the Java Virtual Machine. It also allows use of distributed training of deep-learning models on clusters of Graphics processing units (GPU) and tensor processing units (TPU). Keras has been adopted for use in scientific research due to Python (programming language) and its own ease of use and installation. Keras was the 10th most cited tool in the KDnuggets 2018 software poll and registered a 22% usage.

A subset of machine learning is closely related to computational statistics, which focuses on making predictions using computers, however not all machine learning is statistical learning. The study of mathematical optimization delivers methods, theory and application domains to the field of machine learning. Data mining is a related field of study, focusing on exploratory data analysis through unsupervised learning. In its application across business problems, machine learning is also referred to as predictive analytics.

Machine learning approaches are traditionally divided into three broad categories, depending on the nature of the “signal” or “feedback” available to the learning system:

• Supervised learning: The computer is presented with example inputs and their desired outputs, given by a “teacher”, and the goal is to learn a general rule that maps inputs to outputs.
• Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden patterns in data) or a means towards an end (feature learning).
• Reinforcement learning: A computer program interacts with a dynamic environment in which it must perform a certain goal (such as driving a vehicle or playing a game against an opponent). As it navigates its problem space, the program is provided feedback that’s analogous to rewards, which it tries to maximize.

Other approaches have been developed which don’t fit neatly into this three-fold categorisation, and sometimes more than one is used by the same machine learning system. For example topic modeling, dimensionality reduction or meta learning.

As of 2020, deep learning has become the dominant approach for much ongoing work in the field of machine learning.

Python is dynamically-typed (executing at runtime many common programming behaviours that static programming languages perform during compilation) and garbage-collected (with automatic memory management). It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented and functional programming. It was created in the late 1980s, and first released in 1991, by Guido van Rossum as a successor to the ABC programming language. Python 2.0, released in 2000, introduced new features, such as list comprehensions, and a garbage collection system with reference counting, and was discontinued with version 2.7 in 2020. Python 3.0, released in 2008, was a major revision of the language that is not completely backward-compatible and much Python 2 code does not run unmodified on Python 3. With Python 2’s end-of-life (and pip having dropped support in 2021), only Python 3.6.x and later are supported, with older versions still supporting e.g. Windows 7 (and old installers not restricted to 64-bit Windows).

Python interpreters are supported for mainstream operating systems and available for a few more (and in the past supported many more). A global community of programmers develops and maintains CPython, a free and open-source reference implementation. A non-profit organization, the Python Software Foundation, manages and directs resources for Python and CPython development.

As of January 2021, Python ranks third in TIOBE’s index of most popular programming languages, behind C and Java, having previously gained second place and their award for the most popularity gain for 2020. It was selected Programming Language of the Year in 2007, 2010, and 2018.

An empirical study found that scripting languages, such as Python, are more productive than conventional languages, such as C and Java, for programming problems involving string manipulation and search in a dictionary, and determined that memory consumption was often “better than Java and not much worse than C or C++”. Large organizations that use Python include i.a. Wikipedia, Google, Yahoo!, CERN, NASA, Facebook, Amazon, Instagram.

Beyond its artificial intelligence applications, Python, as a scripting language with modular architecture, simple syntax and rich text processing tools, is often used for natural language processing.