# Train Your Object Recognition Model from Scratch

[**Nikita Beresnev**](https://www.strv.com/blog/authors/nikita) iOS Engineer

---

Machine learning has been around for a while now, but only recently has it begun to accelerate. Why? The reason is pretty simple. People have managed to radically improve the computational power of everyday devices in a relatively short period of time.

The first consumer PC was introduced in 1975. That’s a mere 44 years, and look where we are now. The tech we wear on our wrists or carry around in the pockets of your jeans is much more powerful than the computers available to the public a decade ago.

People are lazy by nature. We tend to invent things we don’t actually need for the sake of saving a few extra minutes or hours of our daily routine. We now have endless power and data, and we continue to automate any task we can. Funnily enough, our laziness yields pretty ingenious results.

For a while, image recognition, text generation or even playing games were believed to be something only humans could do. But we’ve managed to use modern tools to help shake ourselves of full responsibility in these cases, too. Machine learning has already been successful in doing “human” activities — like detecting cancer, preventing car crashes, etc. Machines keep surpassing limitations.

And yet, no matter how powerful they get, all machines understand are 0s and 1s. They will always need a bit of extra help from us. With this article, I want to show you exactly how to provide that help.

## ML OPTIONS IN THE APPLE ECOSYSTEM

In the Apple ecosystem, the seemingly obvious solution is to choose Core ML model. However, there are multiple paths you can take in order to create a model. For now, let's focus on solutions that do not require much machine learning expertise, that simplify development with prepared out-of-the-box solutions with decent performance for typical Machine Learning tasks and that work best with the Apple ecosystem. Good examples are Turi Create and Create ML. However, both come with pros and cons.

**TURI CREATE**

**Pros:**
- More flexible (not tied to the UI)
- Supports more use cases (one-shot object detection, etc.)
- Not tied only to macOS (also supports Windows and Linux)
- Supports various annotation formats

**Cons:**
- Cumbersome installation process

**CREATE ML**

**Pros:**
- Has pretty simple UI
- Comes with the latest XCode

**Cons:**
- Not as flexible as Turi Create (more scenarios are supported by Turi Create, users can see the bounding boxes in the previews, etc.)
- Currently, not much info is available

In this article, we’re going to concentrate mainly on the Turi Create solution and will briefly touch on Create ML.

## TURI CREATE INSTALLATION

**Prerequisites:**
- [Python 2.7.x/3.7.x](https://www.python.org/?ref=strv.ghost.io)
- [Python Virtual Environment](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Jupyter Notebook](https://jupyter.org/index.html?ref=strv.ghost.io)

**Virtual Environment**

We will be using `virtualenv` - a tool that’ll help us create an isolated Python environment, where we’ll be training our model. You can install `virtualenv` by running the following command in your terminal.

```bash
pip install virtualenv
```

To verify the currently installed version, run:

```bash
virtualenv --version
```
Output example:
```
~> 16.5.0
```

If you’re looking for more information about Virtual Environments, please refer to the following links:
- [Docs](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)
- [Installation](https://virtualenv.pypa.io/en/latest/installation/?ref=strv.ghost.io)

**Jupyter Notebook**

In order to create a virtual environment, navigate to a location where you would like to store your project using terminal and run:

```bash
virtualenv TuriSample
```

The created virtual environment `TuriSample` contains three directories: `bin`, `include`, and `lib`, which contains your dependencies, reference to Python used, and other files needed when running the environment.

To activate the environment, run:

```bash
source TuriSample/bin/activate
```

Your prompt should change to indicate the environment is active:

```
(TuriSample) Nick Beresnev:Untracked strv$
```

To install Turi Create:

```bash
pip install turicreate
```

When finished, install Jupyter Notebook:

```bash
pip install jupyter
```

And then open Jupyter:

```bash
jupyter notebook
```

This should open a window inside your browser.

---

## 6-STEP RECIPE

The 6-step recipe is a recommended set of steps required to create your own trained models:

1. Task understanding  
2. Data collection  
3. Data annotation  
4. Model training  
5. Model evaluation  
6. Model deployment

### Task understanding

We need to clearly understand the problem we’re trying to solve, what kind of dataset is required, what the model will be responsible for, etc.

In this tutorial, we’ll work on an object detection task. The trained model will classify ("what") and localize ("where") mango, pineapple, banana, and dragonfruit instances in an image.

### Data collection

To train an image recognition model, we need a **representative** dataset. Use many photos of your objects in different contexts — angles, scales, lighting conditions, backgrounds, and assorted objects. Ensure enough samples per object.

**Example:**  
We prepared some training data, available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection/tree/master/DataSet/images?ref=strv.ghost.io).

### Data annotation

A dataset without annotations is useless. Manually annotate each picture, or use tools like MakeML on macOS or [IBM Cloud Annotation](https://cloud.annotations.ai/?ref=strv.ghost.io). 

An example annotation looks like:

> This is how an annotated image looks:  
> The annotation includes the image path, coordinates, and a label. Coordinates are pixel values for the bounding box center (X, Y), height, and width.

Annotations of images are available in a [CSV file](https://github.com/strvcom/ios-research-ml-object-detection/blob/master/DataSet/annotations.csv?ref=strv.ghost.io).

### Model training

Once data is ready, proceed with training. It may take time depending on the number of images, iterations, hardware, etc. We’ll use a CNN, typically set for around 1,200 iterations. Split data into training (80%) and testing (20%) sets.

Use a Mac with GPU for faster training (up to hours vs. minutes).

To start training, activate the terminal:

- Navigate to your environment folder:  
  `cd ~/Documents/TuriSample`  
- Activate environment:  
  `source TuriSample/bin/activate`  
- Launch Jupyter:  
  `jupyter notebook`

In the notebook, import Turi Create:

```python
import turicreate as tc
```

Load images:

```python
images = tc.image_analysis.load_images("TuriSample/Images")
```

And explore:

```python
images.explore()
```

Load annotations:

```python
annotations = tc.SFrame.read_csv("TuriSample/annotations.csv")
```

Join data:

```python
joined_sframe = images.join(annotations)
```

Split into training and testing:

```python
training_sframe, testing_sframe = joined_sframe.random_split(0.8)
```

### Training a sample model (50 iterations)

```python
model_50 = tc.object_detector.create(training_sframe, max_iterations=50)
```

This process might take some time.

### Comparing models

Evaluate the model:

```python
model_50
```

It outputs model info including class, training time, epochs, accuracy metrics, etc.

For better accuracy, increase iterations (e.g., 300+).

### Alternative route: CreateML

Use CreateML with annotations embedded in image folder, in JSON format, for a more straightforward setup.

Steps:
- Drag image folder into CreateML.
- Set iterations and train.
- Export the model for deployment.

---

## Model evaluation

Use the `evaluate()` method to assess performance with metrics like mean average precision (mAP):

```python
metrics_50 = model_50.evaluate(testing_sframe)
```

Outputs average precision per class and overall mAP.

Upon training longer (e.g., 800 iterations), accuracy improves:
```python
metrics_800 = model_800.evaluate(testing_sframe)
```

---

## Model deployment

Export the trained model:

```python
model.export_coreml("custom_model")
```

It’s saved above your current directory.

## Sample project

The sample app streams camera input, draws real-time bounding boxes with labels and confidence, and allows switching between pre-trained and custom models.

Main files:
- `TrackItemType.swift`
- `VisionService.swift`
- `MLModelService.swift`

`MLModelService.swift` loads/switches models.

Import your `custom_model.mlmodel` into:

`TuriCreate/Models/Trained Models`

Configure your team in Xcode, connect an iPhone, and run — **not** on Simulator (camera needed).

---

## Final notes

Train your own models, run the example app, and adjust for your dataset quality. Our model was trained on few images, so accuracy may vary.

The dataset and code are available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection?ref=strv.ghost.io).

Thanks to [Jaime López](https://www.linkedin.com/in/jaime-andr%C3%A9s-l%C3%B3pez-mora-96b1a910b/?ref=strv.ghost.io) and [Jan Maly](https://www.linkedin.com/in/jan-maly/?ref=strv.ghost.io) for their input. 

We’re excited about advancing using data, analytics, ML, and AI. The future is now.

---

## Sources
- [Turi Create User Guide](https://apple.github.io/turicreate/docs/userguide/?ref=strv.ghost.io)
- [Turi Create GitHub](https://github.com/apple/turicreate?ref=strv.ghost.io)
- [API Documentation](https://apple.github.io/turicreate/docs/api/?ref=strv.ghost.io)
- [WWDC 2019: WWDC 2018 Videos](https://developer.apple.com/videos/play/wwdc2019/420/?ref=strv.ghost.io)