Deep learning is a subset of machine learning that uses artificial neural networks to identify patterns in data. In a deep learning system, large amounts of data are used to train a neural network on the data. Then, new data that was not used in training is used on the trained neural network (“the model”) to predict new results.

In the past year, ChatGPT and other large language models (LLMs) have propelled deep learning to the forefront of public consciousness. These models have been trained on billions of pages of text and images. Then, a user can type a question into the LLM that it has not been trained on. The LLM will then predict an appropriate answer.

LLMs have pioneered a number of powerful techniques for deep learning. Most notably, the “transformer architecture” (https://arxiv.org/abs/1706.03762) has been a game changing technique that helps neural networks focus their attention on the most relevant patterns in training data. (Yannic Kilcher’s YouTube video is a helpful introduction to this paper.)

The Data Ecosystem

All LLMs are powered by copious amounts of data. While AI has dominated headlines, the data ecosystem has also been innovating at an extraordinary pace. There is clear overlap between the technical capabilities of data lakes, data warehouses, AI, ML, and analytics (see The Great Data Debate with Bob Muglia, Tristan Handy, George Fraser, Michelle Ufford for an enlightened discussion). The uber trend here is that collecting and storing data at scale is easier and cheaper than ever.

So if we’re in a world where the cost of storing and retrieving data approaches zero, and we also have increasingly sophisticated models that can take advantage of this cornocupia of data … what’s next?

Data, deep learning, and trends

The Data Ecosystem

Next