Realizing the potential for artificial intelligence to help us better serve our clients, RBC founded Borealis AI, a world-class research institute with a focus on state-of-the-art and ethical AI.
In a recent Harvard Business Review article, Alex LaPlante and Kathryn Hume from Borealis AI discuss the complexities of building machine learning systems in large organizations like RBC and explain why it’s critical to ensure frequent cross-functional communication at every step. Here Alex outlines the key points from this article.
How are business leaders using machine learning today?
Machine learning
is capable of
analyzing millions
of data sets without
being explicitly
programmed
Business leaders are increasingly looking to machine learning to improve business outcomes or create new products and revenue streams. Because machine learning is capable of analyzing millions of data sets within a short time without being explicitly programmed, it has broad application to help organizations manage risk, generate insights, and enhance the client experience. For example, some organizations are currently using it to detect fraud and others to automate trading activities.
What challenges can occur when implementing machine learning?
Machine learning developers may be unaware of the constraints associated with building enterprise solutions, whereas business stakeholders may not to know how these constraints would affect algorithm design choices if developers knew about them. A common refrain is that business and technical teams align on a general problem statement for a machine learning task, and then the technical team goes off and experiments with different algorithms and brings the most promising one back to the business for implementation. But at this point, the business points out that the solution must pass a variety of validation checks, satisfy privacy and fairness requirements, and implementing it will require more engineering resources than the business is willing to commit. The technical team must go back to the drawing board and redesign their algorithm taking these constraints into account. Had both teams aligned on these issues from the get-go, this drawback could have been avoided.
These sound like common project challenges. Are there aspects of machine learning that makes communication and coordination particularly important?
Yes, at the heart of the matter lies the structure of machine-learning algorithms, which use data to learn approximate mappings between inputs and outputs that are useful for a business. With standard software, programmers write specific instructions that execute the same operations every time; the trade-off is that these instructions are limited to what can be articulated in code. With machine learning, by contrast, programmers specify the goal of the program and write an algorithm that helps the system efficiently learn the best input-output mapping from available data to achieve this goal, rather than selecting a particular mapping from the get-go. This approach enables us to tackle scenarios where it’s much harder to write the precise rules (e.g., image recognition, textual analysis, generating video sequences), but the trade-off is that the system’s output can be unpredictable or unintuitive. Decisions on these trade-offs need to be made early on during system development.
At the heart of the matter lies the structure of machine-learning algorithms, which use data to learn approximate mappings between inputs and outputs.
What do organizations need to do to develop machine learning products efficiently?
Project owners need to know what trade-offs and decisions they will face while building a machine learning system, and when they should assess these trade-offs to minimize frustration and wasted effort. In order to build this knowledge, they need to engage the business stakeholder, end user, and governance teams from the get-go, and continue to do so throughout the development process.
Borealis AI breaks the ML development process into five phases:
- Design: Define the problem, determine how to measure business value, and identify business and regulatory requirements that could impact the form the solution to this problem takes.
- Explore: Conduct a feasibility study to determine whether the available data are sufficient to solve the problem, whether the data are biased or imbalanced, and the extent to which the business needs an explainable solution.
- Refine: Train and test the model. Assess its ability to satisfy the business’ needs.
- Build: Implement a production-grade version of the model and integrate it into the business’ existing systems.
- Measure: Document the models ongoing performance stability. Discuss how to manage unexpected errors and whether to scale to new contexts or add additional features.
Depending on the outcome of each phase, going back to a previous one to re-iterate may be necessary. For example, if the feasibility study indicates the data are too noisy, it may be necessary to go back to the design phase to reformulate the problem. Or if model performance degrades over time during the measurement phase, the model may need to be rebuilt to take this issue into account.
Read the full Harvard Business Review article “Managing Bias and Risk at Every Step of the AI-Building Process.”