# Train Your Object Recognition Model from Scratch

[Nikita Beresnev](https://www.strv.com/blog/authors/nikita) iOS Engineer

---

Machine learning has been around for a while now, but only recently has it begun to accelerate. Why? The reason is pretty simple. People have managed to radically improve the computational power of everyday devices in a relatively short period of time.

The first consumer PC was introduced in 1975. That’s a mere 44 years, and look where we are now. The tech we wear on our wrists or carry around in the pockets of your jeans is much more powerful than the computers available to the public a decade ago.

People are lazy by nature. We tend to invent things we don’t actually need for the sake of saving a few extra minutes or hours of our daily routine. We now have endless power and data, and we continue to automate any task we can. Funnily enough, our laziness yields pretty ingenious results.

For a while, image recognition, text generation or even playing games were believed to be something only humans could do. But we’ve managed to use modern tools to help shake ourselves of full responsibility in these cases, too. Machine learning has already been successful in doing “human” activities — like detecting cancer, preventing car crashes, etc. Machines keep surpassing limitations.

And yet, no matter how powerful they get, all machines understand are 0s and 1s. They will always need a bit of extra help from us. With this article, I want to show you exactly how to provide that help.

## ML OPTIONS IN THE APPLE ECOSYSTEM

In the Apple ecosystem, the seemingly obvious solution is to choose Core ML model. However, there are multiple paths you can take in order to create a model. For now, let's focus on solutions that do not require much machine learning expertise, that simplify development with prepared out-of-the-box solutions with decent performance for typical Machine Learning tasks and that work best with the Apple ecosystem. Good examples are Turi Create and Create ML. However, both come with pros and cons.

**TURI CREATE**

**Pros:**
- More flexible (not tied to the UI)
- Supports more use cases (one-shot object detection, etc.)
- Not tied only to macOS (also supports Windows and Linux)
- Supports various annotation formats

**Cons:**
- Cumbersome installation process

**CREATE ML**

**Pros:**
- Has pretty simple UI
- Comes with the latest XCode

**Cons:**
- Not as flexible as Turi Create (more scenarios are supported by Turi Create, users can see the bounding boxes in the previews, etc.)
- Currently, not much info is available

In this article, we’re going to concentrate mainly on the Turi Create solution and will briefly touch on Create ML.

## TURI CREATE INSTALLATION

**Prerequisites:**
- [Python 2.7.x/3.7.x](https://www.python.org/?ref=strv.ghost.io)
- [Python Virtual Environment](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Jupyter Notebook](https://jupyter.org/index.html?ref=strv.ghost.io)

### Virtual Environment

We will be using `virtualenv` - a tool that’ll help us create an isolated Python environment, where we’ll be training our model. You can install `virtualenv` by running the following command in your terminal.

```bash
pip install virtualenv
```

To verify the currently installed version, run:

```bash
virtualenv --version
```
Output example:
```
~> 16.5.0
```

If you’re looking for more information about Virtual Environments, please refer to the following links:
- [Docs](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Installation](https://virtualenv.pypa.io/en/latest/installation/?ref=strv.ghost.io)

### Jupyter Notebook

In order to create a virtual environment, navigate to a location where you would like to store your project using terminal and run:

```bash
virtualenv TuriSample
```

The created virtual environment `TuriSample` contains three directories: bin, include, and lib, which contain your dependencies, reference to Python used, and other files needed when running the environment.

To activate the environment, run:

```bash
source TuriSample/bin/activate
```
Output example:
```
~> (TuriSample) Nick Beresnev:Untracked strv$
```

To install Turi Create, run:

```bash
pip install turicreate
```

When finished, install Jupyter Notebook:

```bash
pip install jupyter
```

And to open Jupyter:

```bash
jupyter notebook
```

This should open a window inside your browser.

---

## 6-STEP RECIPE

The 6-step recipe is a recommended set of steps required for creating your own trained models:

1. Task understanding  
2. Data collection  
3. Data annotation  
4. Model training  
5. Model evaluation  
6. Model deployment

### Task understanding

We must clearly understand the problem we’re solving, what data we need, what our model will do, etc.  

In this tutorial, we will work on an object detection task where the model will classify (the “what”) and localize (the “where”) mango, pineapple, banana, and dragonfruit in images.

### Data collection

To train an image recognition model, we need a representative dataset with many different examples — various angles, scales, lighting, backgrounds — enough to ensure good generalization.

Examples:
We have prepared some training data. Find it in our [repository](https://github.com/strvcom/ios-research-ml-object-detection/tree/master/DataSet/images?ref=strv.ghost.io).

### Data annotation

A dataset without annotations is useless. Annotations include coordinates and labels (tags). You can annotate manually or use tools like MakeML or [IBM Cloud Annotation](https://cloud.annotations.ai/?ref=strv.ghost.io).  

Example:  
This is how an annotated image looks:

*Description explaining the annotation format, including coordinates (center points of bounding boxes in pixels), label, and image path.*

Annotations are available in a [CSV file](https://github.com/strvcom/ios-research-ml-object-detection/blob/master/DataSet/annotations.csv?ref=strv.ghost.io).

### Model training

Once prepared, train the model — this can take time depending on hardware and dataset size. We’ll train a CNN, suggesting around 1200 iterations, but you can adjust.

Most datasets are split into training and testing sets (commonly 80/20). Use:

```python
joined_sframe = images.join(annotations)
training_sframe, testing_sframe = joined_sframe.random_split(0.8)
```

and then create the object detector:

```python
model_50 = tc.object_detector.create(training_sframe, max_iterations=50)
```

This process displays loss and time per iteration.

### Comparing models

Evaluate the model:

```python
metrics_50 = model_50.evaluate(testing_sframe)
```

Results include average precision per class and mean average precision at 50% IoU threshold.

### Alternative route: CreateML

Open CreateML app, load images, set iterations, train, and export model. Annotations should be in the same folder as images, possibly in JSON format.

---

## Model evaluation

Before deployment, assess how well the model performs on unseen data using metrics like mean average precision (mAP) at IoU 0.5.

Use:

```python
metrics_50 = model_50.evaluate(testing_sframe)
```

Results show per-class precision and overall performance, with higher iterations improving accuracy.

---

## Model deployment

Export the trained model with:

```python
model.export_coreml("custom_model")
```

Replace the model in your project, and build with your developer team. Import into Xcode under `Model/Trained Models` as `custom_model.mlmodel`.

Connect an iPhone and run. The app will detect fruits in real time, drawing bounding boxes with labels and confidence levels. The example app includes files:
- `TrackItemType.swift`
- `VisionService.swift`
- `MLModelService.swift`

`MLModelService.swift` loads and switches models; `TrackItemType.swift` defines labels; `VisionService.swift` handles detection and drawing.

Add your custom trained model by drag-and-drop of `custom_model.mlmodel` file into Xcode.

---

## Final notes

Feel free to train your own models and run the app. Keep in mind, trained models are limited by the training dataset quality and quantity.

The app and dataset are available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection?ref=strv.ghost.io).

Big thanks to [Jaime López](https://www.linkedin.com/in/jaime-andr%C3%A9s-l%C3%B3pez-mora-96b1a910b/?ref=strv.ghost.io) and our Head of Machine Learning, [Jan Maly](https://www.linkedin.com/in/jan-maly/?ref=strv.ghost.io).

We're excited for AI and machine learning to continually improve customer experiences and innovate across industries.

---

## Sources:

- [Apple Turi Create User Guide](https://apple.github.io/turicreate/docs/userguide/?ref=strv.ghost.io)
- [Turi Create GitHub](https://github.com/apple/turicreate?ref=strv.ghost.io)
- [Turi Create API](https://apple.github.io/turicreate/docs/api/?ref=strv.ghost.io)
- [WWDC 2019 – Object Detection](https://developer.apple.com/videos/play/wwdc2019/420/?ref=strv.ghost.io)
- [WWDC 2018 – Core ML](https://developer.apple.com/videos/play/wwdc2018/712/?ref=strv.ghost.io)

---

Don't miss anything