What are you interested in?

This book shows you how to do design complex Smart Automation systems on top of Google Cloud for real-world scenarios. The examples presented come from projects we have worked on, showing you the real road bumps we went through and how we worked through them. This book is different from others in the sense that it shows you how we actually work through a solution, not only the final designs and code. Real learning happens through struggle, and that's what you'll get in this book.

Smart Automation (SA) is not a hyped term so many people don't recognize it. Some people misuse the term Artificial Intelligence (AI) to refer to SA projects. Frankly, it sells more, but we prefer to call it Smart Automation. Companies using SA often see significant improvements in their effectiveness and efficiency, while they reduce their operations' risks. In this book we go through three examples of Digital Transformations using Smart Automation in detail and in a way that allows business people to understand its potential.

Everything can be modelled as a time-series. Literally everything. And these time series can be implemented as event streams within software. With the right technology and the right mathematical models, great insights can be produced by analyzing events all around us. In this book go through classical and cutting-edge techniques for time-series forecasting using Python. Another of our courses shows how to leverage this sytem to build a Time-Series Forecasting System that is deployed to Google Cloud.

In this free and short course we show how to build a crawler that detects broken links in a website. We designed it to help us find broken links in our own DELTA LAB website, and figured other people will benefit from it as well, so we decided to make into a small free course. The task at hand is to go through every link in a website and keep track of those that come back as errors when visited, while keeping control of not going into external sites in a way that we end up crawling the whole internet.

In our course "Building a Time-Series Forecasting System" we show how to create the time-series forecasting system that we will use in this course to build a Failure Prevention System. The principles apply all the same with any time-series forecasting system you may want to use, as long as it supports multi-variate time-series forecasts in real-time. By the end of the course, you'll be equipped to use a system like this within your company to reduce your workloads when it comes to fixing your own systems.

Often people who know how to train and test Machine Learning models don't know how to deploy them to real production systems. That's a big knowledge gap that will prevent many from realizing the full value of their models. In this course we show how to not only train and test Machine Learning models, but also how to make the operational for real-world use. By the end of the course, you'll be able to deploy APIs that produce real-time predictions that you can request through HTTP calls.

Often people building systems for clients over-engineer or over-complicate the building of simple dashboards. Often off-the-shelf solutions can be combined, through clever design, to provide clients with the value they need without you having to build everything from scratch. In this course we show you how to build a real-time dashboard that shows predictions coming from Machine Learning systems for time-series forecasting. By the end of the course you will have built a complex, real-time visualization system.

People who build Machine Learning (ML) and Operations Research (OR) models often think of their models as single entities that act in isolation, and they don't realize how multiple models can be combined to produce complex real-time predictions that can dynamically adjust based on how much time a request can be delayed. In this course, we show you how to implement a smart streaming system that uses the Pub/Sub pattern to dynamically plug-in models in while running in production without affecting on-going requests.

Everything can be modelled as a time-series. Literally everything. And these time series can be implemented as event streams within software. With the right technology and the right mathematical models, great insights can be produced by analyzing events all around us. In this book go through the design and implementation of time-series forecasting system with Python, and we deploy it to Google Cloud. Another of our courses shows how to leverage this sytem to build a Failure Prevention System.

Companies often want to know how customers perceive their product while they use it and when they post their experiences online. In this course we will build a system using Python that monitors Twitter posts and extracts the sentiments associated with a product through time. By the end of the course you'll be able to use this tool to keep track of how people feel about live events, company products, the latest sport matches, and any other topic people post on Twitter about.

The standard way of training and deploying Machine Learning (ML) models is with centralized architectures. This means that there are centralized servers, that receive requests from devices and respond with ML predictions. These systems have the disadvantange, as with any other centralized system, that they can be overloaded and that they must have central access to all data. In this course we show how distributed ML systems can be trained and deployed in a way that eliminates some of the problems of centralized architectures.

Often companies manage their inventories using spreadsheets and analyst insights, even when ther are large amounts of data readily available for them to optimize them. Smart companies leverage data publicly available, as well as trends from customer and provider behaviors, to forecast what the optimal inventory size and distribution should be. In this course we show how to build a system that combines various types of techniques to optimize a companies inventory leveraging mathematics and data.

Choosing a name for this site was not an easy task. We went through over 20 different names over a 6 month period, and believe me it was not easy. A name may not me important for many, but for us it's a way of identifying and putting meaning to the work we do. Some people have asked why we chose this name, and in this post we go through some of the evolution for our name and explain the simple reasoning behind the name we decided to stick with, DELTA LAB.

Machine Learning (ML), as any other software implementation, must be tested to ensure it behaves as expected. Even more so in the case of ML, since it produces dynamic outputs based on changing inputs which often had not been seen before by the system, it is important for ML tests to look out for model drift by performing distribution tests, for example. In this post we go through various practical techniques that should be used to test production ML systems and things to look out for.

Often companies develop or buy technology without calculating their Total Cost of Ownership (TCO), which is a big mistake. TCO doesn't only include cloud and development costs, but also the amount of time it will take your teams to learn the new system, and many other factors. These costs combined must be compared against a similar holistic calculation for the benefits, and only then can informed business decisions be made. In this post we go through an example of such calculations for a client of ours.

Some people think that Machine Learning will not scale enough to meet their needs. That's incorrect. As with almost any other decision-making software, Machine Learning can scale as much as needed given the right understanding of business and system constraints, and the right design. First, how much do you really need to scale? Second, what are your success metrics? In this post we go through an example of how to make a system scale to large global workloads. There's a matching course in which we show how to implement this system.

Some people state that Machine Learning is a black-box and that its results can't be trusted because we can't understand them. To some degree they're correct. Although I'd ask, do you really understand how your car works? In this post we go through different ways to make sense out of common Machine Learning models, and how to decrease the lack of understanding for those really complex ones. It's a technical post that references some of our more high-level work on the damage that biased Machine Learning can have.

For complex problems that require quick decisions, some people believe that Smart Automation is not fast enough. They think that because models sometimes take a bit of time to be trained and produce inferences they won't be able to produce sub-second recommendations. That is not correct, and this post we go through a design that can provide very fast decisions in real-time, with the option to receive even better decisions if the system can wait for a bit longer. In one of our courses we show how to actually build this system.

Digital transformations are composed of many small changes in a company's operations which combined produce a whole new way of working. They empower teams with tools and skills from Systems Thinking, Machine Learning, Operations Research, and other disciplines, so that they make smarter and faster decisions by leveraging Smart Automation with data-driven algorithms. To ensure success, these transformations require diversified portfolio strategies and this post we show how to build them.

Smart Automation normally goes through three stages. First, "description" to undertsand the the problem being tackled and the underlying data dynamics. Second, "prediction" to validate forward-looking findings and receive human feedback. Finally, "prescription" to continuously support complex human decision-making using data-driven algorithms. The difficulty level increases exponentially for each subsequent stage. In this post you will learn strategies for each these stages from a business perspective.

Data Science combines human domain knowledge and intuition with programmatic data analysis to find valuable patterns that provide business value. Once relevant dynamics have been identified, these techniques should be automated so that continuous insights can be provided effortlessly to decision makers that rely on this information. Often Data Science content only shows methods to analyze data, but in this post you will also learn, at a high-level, how to operationalize Data Science for Smart Automation.

Data Engineering focuses storing and moving data between processes, just like house water pipelines transport water from streets to showers and back. If you're tackling complex problems and you're not using appropriate tooling, your risk failure increases. Some processes may be senstive to data quality while others may be sensitive to speed, volume, or other properties. In this post you will learn the top tools and techniques for different use cases, and trade-offs between them from a business perspective.

Cloud Engineering uses properties from cloud services, such as scalability and elasticity, to provide you with cost-effective computational power and storage so that you can trade CapEx for lower OpEx to reduce your Total Cost of Ownserhip (TCO). These trade-offs, combined, provide you with the agility and speed you need to tackle the toughest problems. In this post you will learn the top cloud services for different use cases, and the trade-offs between them from a business perspective.

Everything can be modelled as a time-series. Literally everything. And these time series can be implemented as event streams within software. With the right technology and the right mathematical models, great insights can be produced by analyzing events all around us. In this post we go through high-level examples of how these types of systems can be leveraged by companies to materialize significant value. There's a matching course that shows how to actually implement these types of systems.

Operations Research is an area focused on mathematical optimization. It's not hyped as Machine Learning, but it has as much impact on business value for data-intensive companies. At a high-level, it works by providing an objective that will be optimized given a set of resource restrictions. In this post we go through the value of Operations Research from a business perspective without going into its technicalities to keep it useful to a broader audience, and actionable recommendations for these types of projects.

Machine Learning finds patterns in ways that are impossible for humans, specially when large amounts of data are available. Even though it has been around since the 1950's, it has become very hyped during the last decade because its value can now be materialized at scale due to the explosive increase in data production and computational power, and the massive decrease in computational costs. In this post we go through a high-level view of Machine Learning for Smart Automation, and provide actionable recommendations.

People selling Smart Automation often state that large projects with matching budgets are necessary to materialize value. This is absolutely wrong. On the contrary, very large projects should be avoided as they carry unnecessary risk. Digital transformations require iterative experimentation and have high failure rates, which is why you should use diversified strategic portfolios of short, focused, strategic projects that can be quickly killed if necessary. In this post you'll earn how to plan a digital transformation.

Smart Automation leverages feedback loops between humans and machines to efficiently evolve systems and effectively leverage each other's strengths. Humans can think creatively and openly to find novel solutions while machines can efficiently search through enormous amounts of data to find specific patterns. When these two skill sets are combined appropriately, they provide huge value. In this post we go through a high-level view of feedback loops for Smart Automation and the best ways to leverage them.

Most humans have linear intuition and miss out the larger complexities behind systems all around us. We don't understand how a small change in a seemingly unrelated part of a system can have a big impact on another, and we don't see how these changes feed into each other to produce complex behaviors. The study of Dynamical Systems has been around since the 1910's and Systems Thinking has since the 1950's. In this post we go through what these disciplines are and real-world examples of the significant impact they can have on businesses.

Human strengths are in creative thinking to solve complex problems. Machine strengths are in automated data processing for well-defined problems. It's a really bad idea to get humans to do data processing by hand, as well as it's a really bad idea to get a computer to try to solve an abstract problem. You should use the right approach for each problem, and combining human and machine strengths often yields the best results. In this post we go through effective ways of implementing these combinations with actionable insights.

DELTA LAB has a user-driven, evergrowing, knowledge center for Smart Automation. It's user-driven because you can help prioritize content by leaving comments on placeholders that interest you. It's evergrowing not only because new content is consistently published, but also because it is continuously updated. In this post we go through how it works, what it will provide you, and how you can make the most out of it.

Artificial intelligence is advancing at an increasingly fast pace and may soon play an integral role in how our society functions in everyday life. As this development progresses, it's more important than ever to understand how the systems behind this technology work, and how they fail. This is a guest post for Scalable Path in which we dig into the complex issue of bias in Machine Learning with real-world examples, what causes it, and how we can address it moving forward.

Since I wrote this book a couple of years ago, it has received great feedback. R is a high-level statistical language and is widely used among statisticians and data miners to develop analytical applications. Often, data analysis people with great analytical skills lack solid programming knowledge and are unfamiliar with the correct ways to use R. Based on the version 3.4, this book will help you develop strong fundamentals when working with R by taking you through a series of full representative examples, giving you a holistic view of R.