# Train Your Object Recognition Model from Scratch

[Nikita Beresnev](https://www.strv.com/blog/authors/nikita) iOS Engineer

---

Machine learning has been around for a while now, but only recently has it begun to accelerate. Why? The reason is pretty simple. People have managed to radically improve the computational power of everyday devices in a relatively short period of time.

The first consumer PC was introduced in 1975. That’s a mere 44 years, and look where we are now. The tech we wear on our wrists or carry around in the pockets of your jeans is much more powerful than the computers available to the public a decade ago.

People are lazy by nature. We tend to invent things we don’t actually need for the sake of saving a few extra minutes or hours of our daily routine. We now have endless power and data, and we continue to automate any task we can. Funnily enough, our laziness yields pretty ingenious results.

For a while, image recognition, text generation or even playing games were believed to be something only humans could do. But we’ve managed to use modern tools to help shake ourselves of full responsibility in these cases, too. Machine learning has already been successful in doing “human” activities — like detecting cancer, preventing car crashes, etc. Machines keep surpassing limitations.

And yet, no matter how powerful they get, all machines understand are 0s and 1s. They will always need a bit of extra help from us. With this article, I want to show you exactly how to provide that help.

## ML OPTIONS IN THE APPLE ECOSYSTEM

In the Apple ecosystem, the seemingly obvious solution is to choose Core ML model. However, there are multiple paths you can take in order to create a model. For now, let's focus on solutions that do not require much machine learning expertise, that simplify development with prepared out-of-the-box solutions with decent performance for typical Machine Learning tasks and that work best with the Apple ecosystem. Good examples are Turi Create and Create ML. However, both come with pros and cons.

**TURI CREATE**

**Pros:**

- More flexible (not tied to the UI)
- Supports more use cases (one-shot object detection, etc.)
- Not tied only to macOS (also supports Windows and Linux)
- Supports various annotation formats

**Cons:**

- Cumbersome installation process

**CREATE ML**

**Pros:**

- Has pretty simple UI
- Comes with the latest XCode

**Cons:**

- Not as flexible as Turi Create (more scenarios are supported by Turi Create, users can see the bounding boxes in the previews, etc.)
- Currently, not much info is available

In this article, we’re going to concentrate mainly on the Turi Create solution and will briefly touch on Create ML.

## TURI CREATE INSTALLATION

**Prerequisites:**

- [Python 2.7.x/3.7.x](https://www.python.org/?ref=strv.ghost.io)
- [Python Virtual Environment](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Jupyter Notebook](https://jupyter.org/index.html?ref=strv.ghost.io)

**Virtual Environment**

We will be using `virtualenv` - a tool that’ll help us create an isolated Python environment, where we’ll be training our model. You can install `virtualenv` by running the following command in your terminal.

```
pip install virtualenv
```

To verify the currently installed version, run:

```
virtualenv --version
# Output example:
~> 16.5.0
```

If you’re looking for more information about Virtual Environments, please refer to:

- [Docs](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Installation](https://virtualenv.pypa.io/en/latest/installation/?ref=strv.ghost.io)

**Jupyter Notebook**

In order to create a virtual environment, navigate to a location where you would like to store your project using terminal and run:

```
virtualenv TuriSample
```

The created virtual environment `TuriSample` contains three directories: `bin`, `include` and `lib`, which contain your dependencies, reference to Python used, and other files needed when running the environment.

To run the environment, execute:

```
source TuriSample/bin/activate
# You should see:
# (TuriSample) Nick Beresnev:Untracked strv$
```

To install Turi Create, run:

```
pip install turicreate
```

When finished, install Jupyter Notebook:

```
pip install jupyter
```

And then start it with:

```
jupyter notebook
```

This should open a window inside your browser.

## 6-STEP RECIPE

The 6-step recipe is a recommended set of steps required in order to create your own trained models. It consists of:

1. Task understanding
2. Data collection
3. Data annotation
4. Model training
5. Model evaluation
6. Model deployment

### Task understanding

We have to clearly understand the problem we’re trying to solve, what kind of dataset we require, and what the model is responsible for. In this tutorial, we will work on an object detection task. The trained model will classify (the “what”) and localize (the “where”) mango, pineapple, banana, and dragonfruit in images.

### Data collection

To train our image recognition model, we need a **representative** dataset. Variety in examples is key — different angles, scales, lighting conditions, backgrounds, and enough samples of each object. Examples are available in the [repository](https://github.com/strvcom/ios-research-ml-object-detection/tree/master/DataSet/images?ref=strv.ghost.io).

### Data annotation

A dataset without annotations is useless. Use formats like JSON or CSV to annotate images. You can use tools like MakeML or [IBM Cloud Annotation](https://cloud.annotations.ai/?ref=strv.ghost.io) (see [tutorial](https://cloud-annotations.github.io/training/object-detection/cli/2.html?ref=strv.ghost.io)). 

An example annotation includes the image path, bounding box coordinates, and label. Coordinates are in pixels with (x,y) at the bounding box center.

Annotations for all images are available in a [CSV file](https://github.com/strvcom/ios-research-ml-object-detection/blob/master/DataSet/annotations.csv?ref=strv.ghost.io).

### Model training

Once ready, train the model. Expect it to take time depending on data size and hardware. We'll train a CNN; 1200 iterations are suggested initially. Split data into training (80%) and testing (20%) sets.

Use commands like:

```
# Create training and test splits
~> training_sframe, testing_sframe = joined_sframe.random_split(0.8)
```

And then train with:

```
~> model_50 = tc.object_detector.create(training_sframe, max_iterations=50)
```

It might take a few minutes. After training, compare models using:

```
~> model_50
```

### Model evaluation

Use metrics like mean Average Precision (mAP). Evaluate with:

```
metrics_50 = model_50.evaluate(testing_sframe)
```

Interpret results: higher mAP indicates better performance. For example, 50 iterations may give ~17% precision on bananas; more iterations improve accuracy.

### Model deployment

Export the trained model to Core ML:

```
model.export_coreml("custom_model")
```

The model is saved above the current directory. Import it into your sample app’s `Model/Trained Models` as `custom_model.mlmodel`. Ensure your team is set correctly in Xcode. Connect your iPhone (not Simulator) and run the app.

## SAMPLE PROJECT

The app streams live camera input, detects fruits with bounding boxes showing labels and confidence. You can switch between the pre-trained and your custom model via segmented control.

Main files:

- `TrackItemType.swift` — defines classes/labels
- `VisionService.swift` — manages detection in video stream
- `MLModelService.swift` — loads and switches models

Import your trained model into `Models/Trained Models` in Xcode. Build and run on a real device.

## Final notes

Train your model with the provided dataset and code, but recognize limitations — accuracy varies with data quality and capture conditions.

The dataset and app are available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection?ref=strv.ghost.io).

Special thanks to [Jaime López](https://www.linkedin.com/in/jaime-andr%C3%A9s-l%C3%B3pez-mora-96b1a910b/?ref=strv.ghost.io) and our Head of ML, [Jan Maly](https://www.linkedin.com/in/jan-maly/?ref=strv.ghost.io).

We believe machine learning and AI are the future — continuously improving customer experience and transforming industries.

## Sources:

- [Turi Create User Guide](https://apple.github.io/turicreate/docs/userguide/?ref=strv.ghost.io)
- [Turi Create GitHub](https://github.com/apple/turicreate?ref=strv.ghost.io)
- [API Documentation](https://apple.github.io/turicreate/docs/api/?ref=strv.ghost.io)
- [WWDC 2019 Video](https://developer.apple.com/videos/play/wwdc2019/420/?ref=strv.ghost.io)
- [WWDC 2018 Video](https://developer.apple.com/videos/play/wwdc2018/712/?ref=strv.ghost.io)

---

Don't miss anything