Small Data and AI

Moving away from Big Data

Data and data analytics in the form of artificial intelligence are core enablers of companies’ transition to Smart Manufacturing. However, as manufacturers begin to integrate AI solutions into their production lines, data scarcity has arisen as a significant impediment. Consider, for example, an auto parts producer who has had just five or six defects per million parts. For traditional analytic systems, there are not enough examples of defects — that is, the data set is not large enough to properly train a visual inspection machine to detect future flaws when they arise.

Fortunately, recent developments in AI systems have made working with smaller data sets viable, so much so that technology research and consulting firm Gartner predicts that by 2025, “70% of organizations will shift their focus from big to small and wide data, providing more context for analytics and making artificial intelligence (AI) less data hungry.” *

The following techniques are representative of the approaches modern AI solutions are taking to work with smaller data sets:

• Transfer learning - This enables AI to learn from a related task where there is ample data applying that learning to a task with limited data. For example, the machine is exposed to thousands of pictures of scratches collected from a variety of sources. It then transfers that knowledge to identify a scratch in a yet unseen product.
• Artificial data generation – This approach maximizes how much information can be drawn from a small amount of data by changing existing data “artificially” or looking at in in different ways.
• Few-Shot learning – This approach feeds a learning model with a small amount of data, and it relies on the concept that a reliable algorithm can be created to make reliable predictions from minimal data sets.

Manufacturers reticent to embrace AI to increase the efficiency of their operations should consider the recent developments in these powerful technologies. The evolution of AI, particularly regarding working with smaller data sets, has significantly increased its value and accessibility, even for small- and mid-size manufacturers.

As your company looks to innovation in AI and other technologies to stay competitive and grow, Washington Trust is here to help you develop the resources you need to succeed.