Making AI Frictionless with Melio AI 🚀

Accelerating AI/ML Value: Transforming Investment to Impact

Admin — Tue, 15 Sep 2026 17:27:20 GMT

Explore how to derive business impact, competitive advantage, and enhanced decision-making from AI/ML, and learn how to tackle the common challenges in achieving rapid time-to-value.

Photo by GPT4-vision.

In today’s business landscape, the race to leverage AI and ML for competitive advantage is more intense than ever. But here’s the million-dollar question: how do businesses translate AI/ML investments into tangible value swiftly and effectively?

But… what is Value?

Many companies face financial limitations that prevent them from undertaking costly and uncertain data science projects. Consequently, shortening the time to achieve value is a critical goal for data science leaders.

However, the concept of “value” needs clarification 🔍.

Typically, “value” refers to whether there is a willingness to pay for a particular AI model, data pipeline, or set of features. In the context of AI/ML, reducing the time to value means quickly deriving benefits from data and analytics.

AI can deliver value in three main forms:

1️⃣ Business Impact (typically measured in financial terms):

To generate a positive business impact, companies should target manual and repetitive tasks that can be resolved by humans in less than five seconds. These tasks are ideal for AI automation, leading to cost savings and efficiency improvements.

💰 Example: A bank implementing an AI platform that uses NLP to review and interpret commercial loan agreements, saving 360,000 hours of lawyer costs annually.

2️⃣ Competitive Advantage (e.g., market share, customer retention):

To enhance competitive advantage, companies can implement AI to differentiate their products or services, improve customer experience, and innovate faster than rivals, thereby gaining market share and enhancing customer loyalty.

🌐 Example: An online learning platform employing AI-driven learning assistants to help students study more effectively. The Q&A style and automated assessments enhanced student engagement and boosted retention.

3️⃣ Enhanced Decision Making (measured by time saved):

To boost decision-making processes, companies can leverage AI to provide faster, data-driven insights, reducing the time spent on analysis and enabling quicker, more informed decisions.

⏰ Example: A hospital group analyses patient care data and prioritises nursing tasks within the hospital to reduce hospital-acquired infections.

Achieving value more rapidly leads to improved decision-making, enhanced efficiency, and maintaining a competitive edge in the marketplace. Therefore, the ability to swiftly identify use cases and extract value from data science initiatives is essential for companies to thrive in today’s competitive environment.

However, the journey isn’t always straightforward. Various challenges related to business strategies, processes, technology, and data can impede AI/ML projects. To truly accelerate AI adoption successfully, it’s crucial to recognize these potential obstacles early on and develop strategic approaches to overcome them.

Challenge 1: Business

Addressing business challenges is essential and should be the first step before starting any AI initiatives. After all, without a supportive sponsor, there will be no budget or end-users for the application. Here are the three most frequently encountered business challenges:

👩‍💼 Lack of business involvement:

The success of a project hinges on active engagement from business stakeholders. When this participation is lacking, there’s a significant risk of developing solutions that do not align with the broader organisational goals. Receiving constructive business feedback shows a clear interest and value in what is being created and should be treasured and embraced.

Example: A bank implements an AI-powered credit scoring model without adequate input from business leaders. The model, while capable of analysing vast amounts of data to assess credit risk, may overlook crucial factors specific to the bank’s customer base and lending policies.

📊 No baseline to compare to:

Without a well-defined baseline, the actual impact of an AI solution may be unclear. Demonstrating the value of an AI solution involves showing improvements over the current method. If you can’t measure the improvement quantitatively, it becomes challenging to show tangible value and support the implementation of the new approach. A baseline allows you to quantify ROI easily, clarifying business investments in ML projects.

Example: An AI-based medical image diagnosis requires a baseline accuracy rate of a radiologist’s diagnoses to compare against the existing process. Without this baseline, it’s challenging to accurately assess the effectiveness of the model and justify its integration into clinical practice.

🔄 Short-term thinking:

While quick wins are enticing, overemphasising short-term gains without considering long-term scalability and adaptability can lead to the accumulation of technical debt. Striking a balance between immediate results and a strategic, long-term perspective is important for sustaining the success of ML projects.

Example: A machine learning model for customer churn prediction can result in technical debt due to hard-coding design components and minimal documentation. This can result in significant refactoring and cost especially if key people leave the team.

Challenge 2: Process

Process challenges are often the most difficult to address because they require altering established human behaviors. However, no AI initiative can succeed without solid processes for implementation, maintenance, and oversight. Here are the three main process challenges that frequently hinder AI projects.

🔍 Data & Model Governance:

Effective governance is critical in managing data and AI models. Inadequate governance can lead to biased outcomes, raise ethical issues, and obscure decision-making processes in models. Furthermore, neglecting governance until the late stages of the development cycle often obstructs project completion.

Example: A financial institution’s credit scoring model reinforcing biases in historical data could result in discriminatory lending practices.

🤝 Lack of cross-functional teams:

The collaboration between domain experts, engineering teams, and data scientists is crucial for successful ML projects to be developed, deployed, and operationalised. When cross-functional teams are absent, there is a risk of disjointed efforts, hindering effective problem-solving, integration, and adaptation of machine learning solutions within the organisation’s operations.

Example: A business-driven project without data science input may have a flawed scope, while a data science-driven project lacking engineering support struggles with deployment, delaying project completion.

📈 Sustaining model performance:

ML models must continuously adapt to changing data, trends, and business needs to maintain effectiveness. Failure to keep model performance at required levels can lead to degraded model accuracy, resulting in poor decision-making, increasing costs and customer churn through degraded service delivery.

Example: An outdated recommendation system in e-commerce can result in decreased sales and lost market share as it fails to reflect current consumer preferences.

Challenge 3: Technology and Data

AI systems are complex, combining the intricacies of standard software systems with the added challenges of large-scale data and models. This makes operationalizing AI systems a particularly tough challenge. While numerous technology and data hurdles exist, here we highlight the top three that are frequently underestimated.

🔧 Data Quality and Management:

Effective data governance is essential for AI/ML projects. Challenges arise when data are poorly formatted, scattered across various sources, or of low quality. These issues can significantly compromise the accuracy of machine learning models.

Example: In a healthcare analytics project predicting patient outcomes with machine learning, the team faced data quality challenges. The dataset, sourced from multiple places, had diverse formats, incomplete entries, and inconsistent quality. These issues hindered accurate predictive modelling, risking misleading conclusions due to data noise. 🏥

🐌 Technology Stack Inefficiencies:

Each AI/ML project often begins from scratch, leading to a lack of standardisation, duplication of efforts, and a drift in technology stacks. This makes it difficult to maintain diverging models and pipelines and presents challenges in onboarding new team members efficiently.

Example: Managing different versions of machine learning frameworks like PyTorch and TensorFlow, sometimes across on-premises and various cloud environments, can become a logistical nightmare. This fragmentation can significantly delay project timelines and increase the complexity of deployments. ⚙️

🧩 Maintaining Model and Pipeline Integrity:

Keeping machine learning models and their associated data pipelines consistent and up-to-date is a challenge due to the rapidly evolving nature of technology and business requirements. This can lead to discrepancies in model performance and difficulties in ensuring that the entire AI system remains coherent and aligned with current objectives.

Example: An organisation might struggle to update its sales forecasting models in line with new market conditions if its data pipelines are not designed to adapt to changes efficiently. This can result in outdated predictions and missed opportunities. 📊

What then?

Each business is unique. However, we’ve gathered some insights from working with 10+ data science teams.

1️⃣ Share knowledge and REUSE: Sharing and reusing ML assets, as exemplified by a startup’s 30% reduction in development time, directly addresses business challenges by optimising resources and accelerating time-to-market.

On the tech side, it enhances development efficiency and ensures consistency, while in processes, it streamlines workflows and facilitates knowledge transfer. This practice offers tangible solutions to various challenges encountered in AI and ML projects.

2️⃣ Form a great TEAM: Creating a dynamic team with a diverse skill set, as seen in a recent project transitioning seamlessly from data preprocessing to model deployment, addresses business challenges by providing comprehensive expertise.

This approach also tackles tech challenges through adaptability and fosters a culture of continuous improvement to address process challenges efficiently. 🌳

3️⃣ Involve Business: Involving stakeholders throughout development aligns solutions with user expectations, addressing business challenges by staying user-centric and market-relevant. It tackles tech challenges through adaptability and ensures effective communication, mitigating process challenges through iterative improvements. 🔄

4️⃣ Iterative DEVELOPMENT: Starting with heuristics and an MVP addresses business challenges by enabling quick market entry and iterative improvements for customer satisfaction. It tackles tech challenges through optimised model accuracy and resource-efficient development while addressing process challenges through a user-centric and agile approach. 🚀

5️⃣ Focus on VALUE: Prioritising features for a better end-user experience addresses business challenges by boosting monetisation and customer satisfaction. It tackles tech challenges through efficient resource allocation and continuous improvement while addressing process challenges by fostering user-centric development and effective decision-making. 💡

Conclusion

Finally, for every data science, machine learning, and AI use cases, always ask…

“Does it solve a practical problem? Does it create tangible value?”

Remember, successful AI/ML solutions create substantial worth.

. . .

Thanks for reading. If you want to know more about cloud-native tech and machine learning deployments, email us at poke@melio.ai.

What the Ops are you talking about?

Admin — Tue, 15 Sep 2026 17:27:02 GMT

How to choose between DataOps vs MLOps vs AIOps - what are the right Ops for your big data team?

Photo by Freepik from Freepik.

The software industry has been obsessed with the various ops-terms since the popularisation of DevOps in the late 2000s . Ten years ago, software development to deployment has a throw-over-the-world approach. A software engineer develops the app, then throw it over to the operational engineers. The app breaks often during deployment and creates significant friction between the teams.

The DevOps practice came about to smooth out the deployment process. The idea is to treat automation as a first-class citizen to build & deploy software applications.

This approach revolutionised the industry. Many organisations started building a cross-functional team to look after the entire SDLC. The team would set up the infrastructure (infra engineer), develop the app (software engineer), build the CI/CD pipeline (DevOps engineer), deploy the app (every engineer), then monitor and observe the app continuously (site-reliability engineer).

In a big team, different engineers have one primary function. But in smaller teams, one engineer often fills many roles. The ideal is to have many team members able to fill multiple functions so the bottlenecks and key-man dependencies are removed. So in reality…

DevOps, instead of it being a job function, is more practice or culture. It should be adopted at the beginning of building any software

Together with the rise of DevOps, a variety of ops are born.

The various Ops that exist in the software development world. Generated by Autor.

SecOps has security at the core, GitOps strive for continuous delivery, NetOps ensures the network can support the flow of data and ITOps focuses on the operational tasks outside of software delivery. But together, the cornerstone of these ops is derived from the vision that DevOps promises:

To get software out there as fast as possible, with minimal errors.

. . .

Five years ago, the phrase Data is the new Oil became all the hype. Leaders around the world started pouring resources into building Big Data teams to mine these valuable assets. The pressure for these teams to deliver was immense — after all, how can we fail with the promise of the new oil? With the rapid expansion, the analytics teams had also experienced their fair share of grief.

Then we made it all happen.

Data scientists became the sexiest career in the 21st century. We build ourselves up and are sitting in the golden age of data and analytics. Every exec has a dashboard. A dashboard with data from across the organisation and the predictive model embedded. Every customer has a personalised recommendation based on their behaviours.

But now adding a new feature takes weeks if not months. The data schema is a mess and nobody knows if we are using the definition of an active customer from credit team or marketing team. We became wary of rolling models out into production because we’re not sure what it will break.

So the data-centred communities stood together to pledge against inefficiencies seeded from badly managed data processes. Since then, a variety of data-centred ops are also born…

The various Ops that are born in the data-centred teams: DataOps vs. MLOps vs. AIOps. Generated by Autor.

. . .

DataOps 🆚 MLOps 🆚 DevOps (and AIOps?)

*Note: In this article, analytics teams refer to traditional BI teams using SQL/PowerBI to generate insights for business. AI teams refer to teams using Big Data technology to build advanced analytics and machine learning models. Sometimes they’re the same team, but we’re going to keep them separate so it’s easier to explain the concepts.

To understand all these different ops, let’s set the scene on how the data flows through the organisation:

Data is generated by customers interacting with software applications.
The software stores the data in its application database.
The analytics team builds ETLs off these application databases from teams across the organisations.
Analytics teams build reports and dashboards for business users to make data-driven decisions.
The data engineers then ingest the raw data, consolidated datasets (from analytics teams) and other unstructured datasets into some form of a data lake.
The data scientists then build models from these massive datasets.
These models then take new data generated by the users to make predictions.
The software engineers then surface the predictions to the users, and the cycle continues…

We know that DevOps is born because of the friction created between the development and operations team. So, imagine the headache that ensued from a 4-way interface, between the ops, software, analytics, and AI teams.

To explain how the different ops come in solving the processes above, here’s a graph plotting some of the the tasks that each job function performs across the timeline.

A graph of the tasks that each job function performs across the timeline. Generated by Author.

Ideally, the X-Ops culture should be adopted at the inception of the project and the practices implemented throughout. To summarise, this is what each Ops means:

DevOps delivers software faster

A set of practices aims to remove the barriers between the development and operations team to build & deploy software faster. It is usually adopted by the engineering teams, including DevOps engineers, infrastructure engineers, software engineers, site reliability engineers and data engineers.

DataOps delivers data faster

A set of practices to improve the quality and reduce the cycle time of data analytics. The main tasks in DataOps include data tagging, data testing, data pipeline orchestration, data versioning and data monitoring. Analytics and Big Data teams are the main operators of DataOps, but anyone who generates and consumes data should adopt good DataOps practices. This includes data analysts, BI analysts, data scientists, data engineers, and sometimes software engineers.

MLOps delivers machine learning models faster

A set of practices to design, build and manage reproducible, testable and sustainable ML-powered software. For Big Data/Machine Learning teams, MLOps incorporate most DataOps tasks and additional ML-specific tasks, such as model versioning, testing, validation and monitoring.

Bonus: AIOps enhances DevOps tools using the power of AI

Sometimes people incorrectly refer to MLOps as AIOps, but they’re quite different. From Gartner:

AIOps platforms utilize big data, modern machine learning and other advanced analytics technologies to directly and indirectly enhance IT operations (monitoring, automation and service desk) functions with proactive, personal and dynamic insight.

So AIOps typically are DevOps tools that uses AI-powered technology to enhance the service offerings. Alerting & anomaly detection offered by AWS CloudWatch is a good example of AIOps.

💎 Principals not Job Roles

It is a misconception that in order to achieve the efficiency that these ops promise, they need to start with choosing the right technology. In fact, technology is not the most important thing.

“DataOps, MLOps and DevOps practice must be language-, framework, platform and infrastructure agnostic.”

Everyone has a different workflow, and that workflow should be informed by the principals — not the technology you want to try, or the technology that is the most popular. The trap of going technology first will be if you want to use a hammer, everything looks like a nail to you.

All the ops have the same 7 overarching principals, but each with its own slight nuances:

1. Compliance

DevOps typically worries about network & application security. In the MLOps realm, industries such as finance and healthcare often require model explainability. DataOps need to ensure the data product is compliant with the laws such as GDPR/HIPPA.

🔧 Tools: PySyft decouples private data for model training, AirCloak for data anonymisation. Awesome AI Guidelines for a curation of principals, standards and regulations around AI.

2. Iterative Development

This principal stems from the agile methodology, which focuses on continuously generating business value in a sustainable way. The product is designed, built, tested and deployed iteratively to maximise the fail fast and learn principal.

3. Reproducibility

Software systems are typically deterministic: the code should run exactly the same way every time. So to ensure reproducibility DevOps only needs to keep track of the code.

However, machine learning models are often retrained because of either data drift. In order to reproduce the results, MLOps need to version the model and DataOps need to version the data. When being asked by an auditor which data was used to train which model to produce this specific result, the data scientist needs to be able to answer that.

🔧 Tools: experiment tracking tools, such as KubeFlow, MLFlow or SageMaker all have functionalities that link metadata to the experiment run. Pachyderm and DVC for data versioning.

4. Testing

Testing for software lies in unit, integration, and regression testing. DataOps require rigorous data testing, which can include schema changes, data drifts, data validation after feature engineering, etc. From ML perspective, model accuracy, security, bias/fairness, interpretability all need to be tested.

🔧 Tools: libraries such as Shap & Lime for interpretability, fiddler for explainability monitoring, great expectation for data testing.

5. Continuous Deployment

There are three components to the continuous deployment of machine learning models.

The first component is the triggering event, i.e. is the trigger a manual trigger by a data scientist, a calendar schedule event and a threshold trigger?
The second component is the actual retraining of the new model. What are the scripts, data and hyperparameters that resulted in the model? Their versions and how they are linked to one another.
The last component is the actual deployment of the model, which must be orchestrated by the deployment pipeline with alerting in place.

🔧 Tools: most workflow management tools have this, such as AWS SageMaker, AzureML, DataRobot, etc. Open-source tools such as Seldon, Kubeflow KFServing.

6. Automation

Automation is the core-value of DevOps, and really there are a bunch of tools specialised in different aspects of automation. Here are some resources for machine learning projects:

7. Monitoring

Software applications need to be monitored, so does machine learning model and the data pipeline. For DataOps, it’s important to monitor the new data’s distribution for any data and/or concept drift. On the MLOps side, in addition to model degradation, it is also paramount to monitor adversarial attacks if your model has a public API.

🔧 Tools: Most workflow management framework has some form of monitoring. Other popular tools include Prometheus for monitoring metrics, Orbit by Dessa for data & model monitoring.

. . .

Conclusion

Adopt the correct X-Ops culture to speed up the delivery of your data-, and machine learning-powered software product. Remember, principals over technology:

1️⃣ Build cross-disciplinary skills: Nurture T-shaped individuals and teams to bridge the gap and align accountability

2️⃣ Automate early (enough): Converge on a technology stack and automate processes to alleviate engineering overhead

3️⃣ Develop with the end in mind: Invest in solution design upfront to reduce friction from PoC to production

. . .

Thanks for reading. If you want to know more about cloud-native tech and machine learning deployment, email us at melio.ai.

Implementing Custom Authentication in Django

Admin — Tue, 15 Sep 2026 17:26:36 GMT

When all other options fail, make your own.

Photo by pch.vector from Freepik.

Introduction

This tutorial will take you from absolute zero to a fully-working custom authentication system in Django.

Django comes with built-in authentication backends that make it really easy to get started and meet most projects’ needs. There are also a slew of Django apps (such as django-allauth) that have been written to integrate with identity providers such as Google and GitHub.

But what do you do when even these don’t meet your needs?

For example, I was recently attempting to integrate a Django application with a Keycloak backend, and found that existing libraries such as:

either did not work as advertised (though PEBKAC errors are not unlikely here) or did not work in exactly the way I needed. My only option was creating my own custom authentication backend. This sounds like a complicated thing to do, but is actually really simple.

I’ve included a TL;DR right up front if you already know a bit about Django. The code is also on GitHub.

TL;DR - Just Gimme the Answer

In a rush? Here’s the answer:

settings.py

# ./mysite/settings.py  INSTALLED_APPS = [  # ... other apps  "django.contrib.auth",  # ... other apps  "my_custom_authentication", # Or whatever your app is called ]  # ...  AUTHENTICATION_BACKENDS = [  "my_custom_authentication.backends.MyCustomAuthenticationBackend", ]

my_custom_authentication_backend.py

# ./my_custom_authentication/backends/my_custom_authentication_backend.py from django.contrib.auth.backends import BaseBackend from django.contrib.auth import get_user_model # Important for custom User objects   class MyCustomAuthenticationBackend(BaseBackend):  def authenticate(self, request, username=None, password=None):  user = # ... some custom logic such as an API call to an identity provider   # Return None if authentication fails, or a User object if authentication succeeds  return user or None

If the above doesn’t make much sense, follow the complete tutorial below:

Starting the application

Poetry

Let’s start by creating a Poetry (you could simply use pip if you prefer) configuration file for the project. If not installed, please install Poetry.

poetry init

Fill in your details, and choose to not define your dependencies interactively. Once complete, open the pyproject.toml file and ensure that it has content similar to:

[tool.poetry] name = "your-application-name-here" version = "0.1.0" description = "" authors = ["Your Name <your-email@example.com>"]  [tool.poetry.dependencies] python = "^3.10" Django = "^4.0.5"  [tool.poetry.dev-dependencies]  [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"

Then open up a Poetry virtual environment and install the dependencies in it:

poetry shell poetry install poetry update

Check the installation of Django:

python -m django --version # 4.0.5

Starting the Django Project and App

If you don’t already know, Django has the concept of “projects” and pluggable “apps” that you install into the project. Let’s start the project by running:

django-admin startproject mysite .

Note the trailing dot in the above command.

Now we can go ahead and create the django app. We’ll make this application reusable, so go ahead and give it a name that suits your needs. For example, if your ultimate goal is to authenticate with Dex or Keycloak call it something like dex_authentication or keycloak_authentication. As a way of example, I am just going to do an in-memory authentication, so I will run:

python manage.py startapp in_memory_authentication

At this stage, we should already be able to start the Django application. Run:

python manage.py runserver

Navigate to the URL shown (usually http://localhost:8000) and you should see a lovely rocket blasting off:

The running Django application

Understanding Django’s Default User Authentication

If don’t know already and you’re interested how Django would do normal User authentication, check out this great tutorial. Because it’s already covered so well there, I won’t cover it here, and rather just proceed to customizing our user authentication.

Implementing a Login Page

The first thing we’d need to do is include a login page for our Users to authenticate. Go ahead and open the ./mysite/urls.py file. Add the following marked lines to it:

from django.urls import path, include # Added "include" here  urlpatterns = [  path('admin/', admin.site.urls),  path("accounts/", include("django.contrib.auth.urls")), # Added this line ]

What this does is mount the default Views provided in the django.contrib.auth app to a URL of our choice (in this case accounts/).

Let’s run the server again (python manage.py runserver) and this time navigate to http://localhost:8000/accounts/login. Oops! You should get an error page saying:

TemplateDoesNotExist at /accounts/login/

That’s because we need to define an HTML Template for the login page. Do this by creating the following folders in the root folder (i.e. NOT the mysite nor in_memory_authentication folders).

mkdir templates mkdir templates/registration

Then create a login.html file in the templates/registration folder with the following content:

<h2>Log In</h2> <form method="post">    <button type="submit">Log In</button> </form>

One final thing we need to do is configure our Django project to look for templates in this folder structure. Open up the settings.py again, and look for the TEMPLATES constant:

TEMPLATES = [  {  # ...  'DIRS': [],  # ...  }, ]

Change the DIRS value from an empty list to:

TEMPLATES = [  {  # ...  'DIRS': [BASE_DIR / "templates"],  # ...  }, ]

This instructs our Django project to look for templates in folders called templates.

Restart the server and navigate to http://localhost:8000/accounts/login again. You should see a beautiful login form:

The login form. Hey, if you wanted a tutorial on CSS you came to the wrong place.

Attempting to login with details will not work, obviously, for a number of reasons:

We have not yet migrated our database, so have no User table
Even if we had migrated our database, we have not created any User records.

Let’s correct the first problem now. First, stop the server and then run the following command to migrate the database:

python manage.py migrate

Huh? You may be wondering at this stage where the definition of our User table comes from. It comes from the django.contrib.auth app which is by default in the INSTALLED_APPS in settings.py.

Rerun the server, and try to login with any details. You should see a little message reading: Please enter a correct username and password. ...

Writing a Custom Authentication Backend

Let’s now assume that we want to always allow access to the site with a hard-coded user with the credentials:

Username: let_me_in
Password: please

Of course you would never want to do this, but it does illustrate how to create the custom Django authentication backend.

First, let’s create a new folder in our ./in_memory_authentication folder called backends. Like any Python module, it will need a __init__.py file, so create that, and go ahead and create a file for the actual backend. I called mine ./in_memory_authentication/backends/in_memory_authentication_backend.py.

Let’s populate this file now with the following:

# ./in_memory_authentication/backends/in_memory_authentication_backend.py  from django.contrib.auth.backends import BaseBackend   class InMemoryAuthenticationBackend(BaseBackend):  def authenticate(self, request, username=None, password=None):  return None

You will notice that I’m extending the BaseBackend auth backend. It has a single function authenticate that takes in the request, as well as username and password. The function should respond in the following way:

Should return None if the authentication attempt fails.
Should return an instance of the User model if the authentication attempt succeeds.

Let’s export this in the newly-created __init__.py file:

# ./in_memory_authentication/backends/__init__.py  from .in_memory_authentication_backend import InMemoryAuthenticationBackend

Let’s also “register” this backend with our Django app. Open up the settings.py file and make the following two changes:

INSTALLED_APPS = [  # ... other apps  "django.contrib.auth",  # ... other apps  "in_memory_authentication", # Install our new app here ]  # ...  # Add this to the bottom of the file:  AUTHENTICATION_BACKENDS = [  "in_memory_authentication.backends.InMemoryAuthenticationBackend", ]

Now if we attempt to login, our custom authentication backend will be called. Of course, we still won’t be able to login since we are always returning None from our authenticate method.

Let’s change that now.

Open up ./in_memory_authentication/backends/in_memory_authentication_backend.py and alter it to:

# ./in_memory_authentication/backends/in_memory_authentication_backend.py import uuid  from django.contrib.auth.backends import BaseBackend from django.contrib.auth.models import User   class InMemoryAuthenticationBackend(BaseBackend):  def authenticate(self, request, username=None, password=None):  # Replace this silly logic with whatever you need:  if username == "let_me_in" and password == "please":   # Create a new user  new_user = User(username=uuid.uuid4().__str__())  new_user.set_unusable_password()  new_user.save()   return new_user  else:  return None

This is some silly logic, and you should replace it with what you need, but it creates a new User and returns that user, thus logging them in. You should replace this with things like a call to a Dex or Keycloak API, for example.

If you now login with the credentials let_me_in and please, you should successfully authenticate.

Of course, we have not built any pages after the user logs in, so you should get a 404 Page not found error at this stage.

Catering for Custom User Classes

You’ll notice above that I am returning an instance of the django.contrib.auth.models.User class. However, not all Django projects will use this, and it is therefore better to make use of the get_user_model() function to find what class this project uses. Further, we can extract the finding of the user into a method:

# ./in_memory_authentication/backends/in_memory_authentication_backend.py import uuid  from django.contrib.auth.backends import BaseBackend from django.contrib.auth import get_user_model   class InMemoryAuthenticationBackend(BaseBackend):  def authenticate(self, request, username=None, password=None):  return self.find_user(username, password)   # Replace this silly logic with something better, such as an API call to  # an identity provider:  def find_user(self, username, password):  if username == "let_me_in" and password == "please":  user_klass = get_user_model()  new_user = user_klass(username=uuid.uuid4().__str__())  new_user.set_unusable_password()  new_user.save()  return new_user  else:  return None

That’s it! We’ve successfully created and used our own custom authentication backend in Django.

Launching Into Uncertainty - 6 Smart Steps to Future-Proof Your ML Models

Admin — Tue, 15 Sep 2026 17:26:17 GMT

Your machine learning model is going to fail in the pandemic. Here’s how to mitigate the damage.

Photo by pch.vector from Freepik.

There have been many articles focusing on how machine learning can and is helping during the pandemic. South Korean and Taiwanese governments successfully demonstrated how they used AI to slow the spread of COVID-19 . French tech company identifying hot spots where masks are not being worn, advising where governments can focus on education. Data and medical professionals collaborating to index medical journal papers on COVID-19.

Participation from the data science community is inspiring and the results are outstanding.

But this post is about the opposite. I am not here to tout the successes of machine learning. I am here to cast a little doubt on all its glory and wonders. I’m here to ask the question:

Are you watching your machine learning models during this uncertain time?

And why not?

Photo by Stephen Dawson from Unsplash.

Real-life data often diverges from training data. From the day you release the accurate, stable model into production, its performance has been degrading every day as your customers evolve. With COVID-19, it is as if the model is released from the safe, incubated Petri dish to the wild wild west.

With the policy changes, the interest rate drops, and the payment holidays, the sudden data shift can have a profound impact on your model. It’s okay to have a decrease in model accuracy, but it is not okay to be ignorant of the changes.

It’s okay to have a drop in model accuracy, but it is not okay to be ignorant of the changes

For the past couple of weeks, I have been involved in impact assessment for these machine learning models. It was overwhelming initially, so I gathered some practical steps in the hopes that other teams can approach the process with a bit more confidence and a bit less panic:

Photo by Stephen Dawson from Unsplash.

. . .

Step 1: Identify affected use cases and stakeholders

Icon from Eucalyp, adapted by Author.

Not all use cases will be affected by the pandemic, like not all models will suffer from data or concept drift .

The first step in any software assessment is the same as the first step in any software development: ask questions.

Ask a bunch of questions.

If you have many on-going use cases and not sure which to tackle first, begin by asking these:

❓ Am I modeling human behavior? (i.e. items purchased during Easter, home loans down payment)

❓Did I have any inherent assumptions about the government or corporation policies that have changed during the crisis (i.e. travel bans, payment holiday from the bank)

Once you have identified the use cases, then pinpoint the stakeholders affected by these use cases. The stakeholders are often subject matter experts in the domain, so they may be able to assist you in step 2 below.

Draw a decision tree for yourself.

If your system touched a human or a policy, carry on with Step 2.

Otherwise, move on.

Step 2: Assess the risks and exposures for when things go wrong

Icon from Eucalyp, adapted by Author.

This pandemic is a classic case of a black swan event. It’s a rare and unexpected virus outbreak with severe consequences.

So the chances are, things are going to go wrong, but how wrong?

Things are going to go wrong, but just how wrong?

Scenario analysis can help us navigate these extreme conditions.

Speaking to the SME (subject matter experts) from the business is incredibly valuable in this step. They often have many questions they want answers to, and data scientists would analyze the data to generate insight packs to these questions. This is the time when these questions can help you interrogate your models.

As an example,

“What happens to our bottom line if real estate sales are decimated?”

Well, this is the time to test it out. If you are modeling the revenue generated from home loans, then examine the data when home loan sales are low (winter months in a summer vacation town). From there, tweak it with the suggestions from the SME and generate a fake input dataset and run it through your model. This can help you assess the areas when the model is at risk.

Giving a quantitative number can help businesses make tough decisions. Even though we feel like we don’t have enough information to provide a prediction, decisions still have to be made. Loans still need to be granted and toilet papers still need to be shipped. Instead of pushing the responsibilities further along the chain, we should focus on doing the best we can.

Give a prediction with all the asterisks attached to it.

If the impact is large, proceed to step 3 for a thorough examination.

Otherwise, move on.

Step 3: Check your entire solution for vulnerabilities

Icon from Eucalyp, adapted by Author.

Machine learning solutions are more than the model itself. There is the data engineering pipeline, inference, and model retraining. It is important to scan the entire solution end-to-end to find vulnerabilities.

Machine learning solution is more than just the model

Here are some areas you can look at:

⏩ Will a bulk quantity break your pipeline? Some supplier’s predictive algorithms were broken by a sudden change in order quantity. On the other hand, some fraud detection systems were overwhelmed by false positives.

⏩ Has the baseload on your infrastructure changed? Some models are less relevant and some are even more important, shifting your computing resources can even out the cost.

⏩ Investigate the global and local feature-importance of your model. Are your top features sensitive to the pandemic? Should they be? There are many articles about explainable AI, and in the time of the unknown, inference explainability could give the transparency people need.

Define checks throughout your entire pipeline.

By the time you reached Step 3, you already determined that your solution is affected by the crisis. So carry on to Step 4.

There is will be no moving on from this point on, only moving forward.

Step 4: Set up automated alerting and healing

Icon from Eucalyp, adapted by Author.

If you are reading this article, you probably don’t have monitoring systems in place. Otherwise, you would be setting up 500 Jira tickets and be on top of your next steps.

Deploying a machine learning model is like flying an airplane

☑️ : Having no logging is like flying without a Blackbox. Any terrorist can hijack the plane without any consequences.

🖥️ : Having no monitoring is like flying blind in new airspace. The pilot has no idea where he’s going, and neither do you.

🌪️️: Having no resilient infrastructure is like flying with an unmaintained engine. The ticket is cheap but the plane could go down at any time.

Photo by Kaotaru from Unsplash, edited by melio.ai

You won’t board a plane with a blind pilot and a broken engine, why would you deploy a model without logging, monitoring, and resilient infrastructure?

Let’s set aside monitoring, what about healing?

. . .

Many matured machine learning solutions have automated retraining built into the system. But in these extreme events, it may be worth taking a look at the refreshed models. Are they using the latest data for the prediction? Should they be?

Automatically redeploying the refreshed models could be risky when facing the unknown. A human-in-the-loop approach could be both an interim and a strategic solution.

1️⃣ Setup automated retraining and reporting (with business KPI as well as the spot checks defined in Step 3).

2️⃣ Alert a human for approval, with more stringent alerting criteria during the pandemic period.

3️⃣ If Steps 1–3 were conducted rigorously, and the risks and exposures acknowledged, then we may be ready to redeploy the models.

Monitor data and model drift and redeploy if necessary.

If you are still not sure, set-up canary deployment and redirect a small portion of your traffic to the new model. Then slowly phase out the old one when the new model is stable.

Step 5: Unpack and communicate future challenges

Icon from Eucalyp, adapted by Author.

The COVID-19 impact ranges from short-supply of toilet paper to a meltdown of the global economy. Most if not all industries are affected, either positively or negatively.

The abnormality may become the new norm

Long term effects such as remote working, a surge in bankruptcy, structural unemployment as well as travel restrictions are unclear. When the historical data is not applicable in this unprecedented situation, the “abnormality” may become the new norm. This forces us to go back and unpack the fundamental business question and rethink many assumptions.

⏩ Do we include the pandemic data into future model training?

⏩ Do we include new features that better reflects the current reality?

⏩ Should we tune the model sensitivity when major disruptions occur?

⏩ Do we need to re-evaluate the model metrics chosen before the pandemic?

⏩ Do we revert all the above when things “return to normal”?

There are more questions than answers at this stage. It would be premature to blindly recommend “best practices”.

The only way is to stay curious, keep communications open, acknowledge risks, and take it one step at a time.

Step 6: Build fallback systems to mitigate the impact

Icon from Eucalyp, adapted by Author.

Have you wondered why dung beetles don’t get lost?

These tiny insects use a combination of guiding systems to help them achieve the highest precision possible. When one system is not reliable, they switch to use another .

When you don’t have any data, you have to use reason ~Richard Feynman

The famous physicist and Nobel Laurette Richard Feynman, on investigating the NASA Challenger program disaster said: “When you don’t have any data, you have to use reason.” (extracted from Range by David Epstein , Chapter 9).

This particularly rings true when machine learning depends on historical patterns. When historical patterns are not reliable, alternative strategies need to be called upon:

⏩ Unsupervised or semi-supervised approaches can help redefine the problem space. Create new segmentations for existing users based on their new behaviors. A past outdoor junkie could be the now mindful yogi.

⏩ Stress-testing: the scenario analysis framework set up in Step 2 can be used to load various extremes of input data.

⏩ Ensemble or Choose: Ensembling is popular owing to its ability to create a strong learner by combining multiple weak learners. Sometimes, that may not be necessary. It is possible to have multiple models for one use case. The engineering pipeline can detect changes or attributes, and use the most appropriate model for inference.

Or all the above, with the human-in-the-loop.

And that is why ladies and gentlemen - AutoML is not going to replace data scientists. Be the voice of reason. Chat to the SMEs. Keep calm and carry on.

. . .

Navigating changes in this uncertain time is challenging. Nobody is certain of anything, and it can be daunting to make a prediction. Following these steps can guide you through unforeseen problems:

1️⃣ Go back to fundamentals, what is this use case about?

2️⃣ Question & explain your model

3️⃣ Spot check your entire solution

4️⃣ Set up logging & monitoring framework

5️⃣ Retrain and redeploy

6️⃣ Future-proof yourself by staying curious

7️⃣ Be the voice of reason, keep calm and carry on

. . .