Thomas Hardy - Fast.ai - Part 1.2: Deployment

This lesson walked through the workflow of deploying a machine learning model using Hugging Face Spaces. You can find the dog/cat classifier I deployed to Hugging face here (apparently dragons are 93% cat…).

Chapter 2 Questionnaire:

Where do text models currently have a major deficiency?

Text models can generate context-appropriate text, but have no guarantee of correctness. Thus, they are capable of generating text that sounds plausible and accurate but is in fact incorrect. This is dangerous as the unobserved eye may not be able to distinguish or evaluate between what is factually accurrate and what is not.
What are possible negative societal implications of text generation models?

Text models can be abused to generate and spread fake content and disinformation. Additionally, in cases with biased training data, text models may propagate theses biases in its output.
In situations where a model might make mistakes, and those mistakes could be harmful, what is a good alternative to automating a process?

Lots of manual oversight and intervention. Monitoring the model - whether by human or via a system - would be helpful.
What kind of tabular data is deep learning particularly good at?

Deep learning does particularly well at tabular data that includes high-cardinality categorical columns and columns containing natural language.
What’s a key downside of directly using a deep learning model for recommendation systems?

Generally, deep learning recommendation systems only suggest products a user might like rather than recommendations that are helpful. As a result, you get recommendations for products you’re already aware of or have already consumed.
What are the steps of the Drivetrain Approach?

There are 4 steps to the Drivetrain Approach: - define an objective - determine the actions necessary to achieve the objective - what inputs can we control - determine the data necessary to achieve the objective - what data can we collect
- build a model to determine the best acctions to take to reach the objective
How do the steps of the Drivetrain Approach map to a recommendation system?
1. The objective is to drive sales by recommending items to the user that they would enjoy and purchase. That they otherwise would not have purchased without the recommendation.
2. The actions necessary are to rank recommendations (presumably based on relevancy to the user’s consumption history/patterns)
3. The data necessary is the consumption history/patterns of many customers, and how specific recommendations perform given the consumption data. Data must be collected on recommendations that generate sales. To do so many randomized experiments must be conducted to collect data on a wide range of recommendations.
4. Build a model to suggest recommendations to the user, measure the success of the recommendations.
Create an image recognition model using data you curate, and deploy it on the web.

Done
What is DataLoaders?

DataLoaders is a class from the fastai package that stores data and provides utilities to transform data into a form that is suitable for the model.
What four things do we need to tell fastai to create DataLoaders?
- what kind of data we are working with
- how to get the data (i.e. path)
- how to label the data (labelling function)
- how to construct the validation set
What does the splitter parameter to DataBlock do?

The splitter parameter determine how the data should be split between training and validation.
How do we ensure a random split always gives the same validation set?

Specify a seed number for the random split. This will cause for the random split to select the same subset of data every time.
What letters are often used to signify the independent and dependent variables?

x for independent variable(s), y for independent variable(s).
What’s the difference between the crop, pad, and squish resize approaches? When might you choose one over the others?

Crop resizes an image by cutting out any part of the image outside the specified size. This can cause the resultant image to have cut out key visual features.

Pad resizes an image by keeping the entire image and respecting its aspect ratio. If necessary, padding is added to the top/bottom of the image (typically with black bars) to ensure that the resized dimensions are achieved. This can result in the image being very small and less detailed.

Squish resizes an image by squishing the entire image into the specified size. This can distort the original aspect ratio.

The resize approach you choose is dependent on the problem at hand and the data you have.

A potentially better approach is RandomResizedCrop which crops a random region of the image. Consequently, for each epoch the model will see and learn from different regions of image.
What is data augmentation? Why is it needed?

Data augmentation is a technique to used to create many augmentations of the same piece of data. The idea is to train the model on many variations of the same data such that it can learn more generalized patterns (i.e. that many variations of an image still depict/represent the same thing).
Provide an example of where the bear classification model might work poorly in production, due to structural or style differences in the training data.

If the model is trained on well-lighted, stock images of bears, the model may perform poorly if the data it receives in production is dark and low-quality/resolution. This is a key problem where the production data and training data are not aligned (i.e. the data the model sees in production is different from what it saw during training. The training data is not representative of the data the model will see in production). This problem is called out-of-domain data.
What is the difference between item_tfms and batch_tfms?

item_tfms and batch_tfms are both functions to perform transformations on data (typically training data). The difference between the two is that item_tfms performs transformations on one item at a time on the CPU, while batch_tfms performs transformations on a batch of items in parallel on the GPU.

What is a confusion matrix?

A confusion matrix is a matrix representation of how your model performed on a classification task (i.e. predictions made vs correct lables). The rows of the matrix represent the actual labels while the columns represent the predictions. Predictions/labels matche on the diagonal, off-diagonal elements represent mispredictions (i.e. incorrect prediction for the given label). The confusion matrix is a useful tool for interpretability. It provides visual insight as to how the model predicted, where it predicted correctly and incorrectly (i.e. where the model was confused).
What does export save?

export saves the trained model (architecture + trained parameters) into a file along with its DataLoaders (any transformations etc.).
What is it called when we use a model for making predictions, instead of training?

When we use a model for making predictions, we are using the model for inference.
What are IPython widgets?

IPython widgets are like UI elements, useful for creating a web UI in a Juypter notebook.
When would you use a CPU for deployment? When might a GPU be better?

A CPU would be a good choice for deployment if you expect your model to:
- perform inference a single item at a time (i.e. no need for parallel operations)
- you have a small model
- you don’t expect huge usage (not expecting huge workload)
A GPU might be better basically in the opposite conditions:
- performing inference in batches
- large model
- large workload
What are the downsides of deploying your app to a server, instead of to a client (or edge) device such as a phone or PC?

Users incur network latency and you are responsible for ensuring that the server handles the workload sufficiently. You also pay out of pocket for running the server.
What are three examples of problems that could occur when rolling out a bear warning system in practice?
- false alerts (warnings for bears when there are no bears)
- missed alerts (not warning for bears where there are bears)
- misclassified alerts (warning for non-bear objects)
- handling low-light images (i.e. night-time)
- handling low-resolution images
- model’s inference is too slow to be useful
What is out-of-domain data?

Out-of-domain data is data that is fundamentally different to the data your model was trained on, thus is out of the model’s domain of knowledge.
What is domain shift?

Domain shift is when the data that your model receives gradually changes over time such that the data it was initially trained on is no longer relevant.
What are the three steps in the deployment process?
1. Manual oversight and human intervention
2. Gradual, small-scoped rollout (i.e. specific small areas initially, then gradually grow)
3. Expansion, monitoring systems are incorporated