Artificial intelligence is no longer just the stuff sci fi movies are made of. It’s quickly filtering into our everyday lives thanks to increasingly sophisticated machine learning applications.
Andrew Schwartz is Chief Technology Officer for software development startup Catchi Inc, and an instructor for DevelopIntelligence. He’s building an AI-powered real estate app that will bring machine learning and AI to individual cell phones to match prospective renters and buyers lifestyles, desires and budget with available options. It’s practical and useful, yet the technology behind it is quite creative, as it’s never been done before.
So, if you’re waiting anxiously for cyborg technology, digital empathy and robots as friends, you may have to wait a bit longer. AI is more foundational in nature, which is why many organizations have aggressively adopted predictive analytics. Meaning, they use machine learning to infer customer needs and make it easier for sales and marketing teams to individually target potential customers.
As an example, when someone downloads a white paper from a corporate website, data analysis and machine learning algorithms cooperate to infer where the downloader works, match that information with transaction information in corporate databases, and figure out how likely that company is to purchase specific products. Then AI processes will layer on context information automatically so a human sales rep can individually target the right people and sell the most profitable product bundles.
Full AI acts interchangeably with us humans. In enterprise situations, we’re not there yet, but the race is on. Therefore, it’s important for technical talent to learn more about AI technology and its application.
AI Now – AI Soon
Why? Strong AI applications generally involve organic developments from within the engineering organization, Schwartz said. “One off product management requests won’t create AI applications,” he explained. “The engineering organization has to understand what data they’re sitting on and how to use it.” Machine learning depends on a consistent flow of data throughout the value chain, and an engineering team that classifies, marks, and builds features from that data.
Let’s say engineers are programming a SaaS server farm for an enterprise software provider. They’re asked to use AI to automatically improve machine efficiency and reduce the cost of shifting data as machines fail or overload. Schwartz said a good AI solution will generally leverage all the underlying system data and context information, whereas a standard engineering pipeline will only mark and save a small fraction of exception or state information in one-off log messages.
As AI systems provide richer features, Schwartz said you need an organization-wide understanding of what data to archive, how to catalog it, and how to feed it in into the right models. Over time, we may also encounter some interesting confidentiality problems, even in cases where we think that privileged data has been aggregated and obfuscated.
“There are a lot of challenges in data management, massaging and collection, cleansing and securing to simultaneously maintain security and privacy while still enabling rich analysis,” Schwartz explained.
The first step in teaching AI involves helping students understand pure data management and data cleansing pipelines. Machine learning algorithms leverage a lot of data, and people need to understand how to work with these large data sets. “When I teach AI or machine learning classes I also help students understand information content,” he said. “Then I put that in a standard, day-to-day context to help them understand how to think about information in general, and how to match that information to different algorithms.”
Schwartz said it’s often helpful to students to think in terms of visual systems before discussing more abstract computer systems. “I find that people very easily grasp the visual. From that you can move on and talk about text analysis, predictive analytics and things like that.” Then, when someone in an organizational context provides requirements, engineers can understand the relationship between what they learned in class and that particular business requirement
The Value of Understanding
Schwartz said the goal in teaching this kind of subject matter is to help students understand the fundamentals, and to give them a framework for ongoing learning. To that end, DevelopIntelligence offers a two-day course, Machine Learning Data Foundation, and a three-day course, Machine Learning Algorithms and Applications. Both are 40 to 60 percent lab-based. Students come to class, work intensively in groups solving different problems, doing analyses and writing code. “We have ongoing conversations to gauge what students understand, their gaps and to communicate very specific information to facilitate them solving real problems,” he explained.
Students are usually software engineers who are actively working on data intensive applications, and they often bring ideas about applying AI in their current projects with them into class. They’re either working on, or about to join, a machine learning project, and are either seeking architectural help with these projects, or they want to explore the underlying concepts. Sometimes, they’re engineering or product managers working to roll AI into their products or teams.
“Students already working on machine learning projects often have working knowledge of some or all of the techniques but may want to learn more of the theory,” Schwartz said. “They might not understand how to put it together, or how to think about the data flow that you need.”
Students with an educational background in software engineering but who are not currently writing code mingle with students who are currently writing machine learning code in the same class. This is useful because instructors can match up each group’s needs and discuss practically how information flows through an organization, and how they could modify it to make better machine learning products.
“For example, if I have a product manager and a hands on software architect in my class, they may not yet realize that the way the requirements come in doesn’t make it automatic for the background data to be preserved,” Schwartz said. “Or, even if preserved, the important data elements may not be stored in an accessible manner for machine learning. It’s a way of setting up a conversation between product management and system architecture after the class to make the organization AI-ready.”
Schwartz said because we’re still very early in the enterprise roll out of machine learning and artificial intelligence, training is largely focused on getting engineers up to speed on existing tools to perform predictive analytics. “The future is probably going to be a lot more like self-driving cars,” he explained. “Meaning, there’s a transition you can see coming as enterprises roll out closed loop optimization applications. In this world, the software learns from the data as it comes in, and automatically improves the end user experience. That is a very different software life cycle.”
It’s something to look forward to.