> For the complete documentation index, see [llms.txt](https://degirum.gitbook.io/pysdk-user-guides/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://degirum.gitbook.io/pysdk-user-guides/pysdk-0.16.0/pysdk/user-guide-pysdk/setting-up-an-ai-server.md).

# Setting Up an AI Server

Read this page if you'll host an AI server or perform inference with a local server. If you plan to use the AI Hub for inference, go to Loading an AI Model to learn about loading models.

## Starting an AI Server

PySDK can be used to configure and launch an AI server. The AI server allows you to run AI inferences on this AI server host initiated from remote clients.

You can start AI server process multiple ways:

* running AI server from a terminal directly on the host OS.
* running AI server as a Linux system service.
* running AI server with a Docker container.

### Terminal

To run the PySDK AI server from a terminal, perform the following steps:

{% stepper %}
{% step %}
**Create or select a user with administrative rights**

Choose a user with administrative rights on the host.
{% endstep %}

{% step %}
**Set up a Python virtual environment**

For convenience and future maintenance, install PySDK in a Python virtual environment. Ensure Python and PySDK are installed in the virtual environment.
{% endstep %}

{% step %}
**Create a directory for the local model zoo**

Create a directory for hosting a local model zoo.

{% code overflow="wrap" %}

```bash
mkdir /home/<your_username>/zoo
```

{% endcode %}

{% endstep %}

{% step %}
**Download models to the local model zoo**

Download the models from the DeGirum AI Hub Model Zoo to the directory created earlier:

{% code overflow="wrap" %}

```bash
degirum download-zoo --path /home/<your_username>/zoo --token "token string" [--url "model_zoo_url"]
```

{% endcode %}

* `"token string"`: Your AI Hub access token from the [DeGirum AI Hub](https://hub.degirum.com).
* Optional model\_zoo\_url: The URL for the model zoo in the format `"https://hub.degirum.com/<organization>/<zoo>"`. If omitted, the public model zoo will be used.
  {% endstep %}

{% step %}
**Start the AI server**

Launch the server with the following command:<br>

{% code overflow="wrap" %}

```bash
degirum server --zoo /home/<your_username>/zoo
```

{% endcode %}

The server runs until you press `ENTER` in the terminal. By default, it listens on TCP port 8778. To specify a different port, use the `--port` argument:

{% code overflow="wrap" %}

```bash
degirum server --zoo /home/<your_username>/zoo --port <your_port>
```

{% endcode %}
{% endstep %}
{% endstepper %}

### Linux Service

To automate the server launch so it starts on system boot, configure it as a Linux service:

{% stepper %}
{% step %}
**Complete the terminal setup steps**

&#x20;Follow all steps in [Starting AI Server from Terminal](#starting-ai-server-from-terminal) except for launching the server.
{% endstep %}

{% step %}
**Create a systemd service configuration file**

Create a file named `degirum.service` in the `/etc/systemd/system` directory. Use the following template:

{% code overflow="wrap" %}

```ini
[Unit]
Description=DeGirum AI Service

[Service]
WorkingDirectory=/home/<your_username>/
ExecStart=<path_to_python> -m degirum.server --zoo /home/<your_username>/zoo
Restart=always
RestartSec=10
SyslogIdentifier=degirum-ai-server
User=<your_username>

[Install]
WantedBy=multi-user.target
```

{% endcode %}
{% endstep %}

{% step %}
**Start the service**

Start the service using `systemctl`:

{% code overflow="wrap" %}

```bash
sudo systemctl start degirum.service
```

{% endcode %}
{% endstep %}

{% step %}
**Check the service status**

Check the service status using `systemctl`:

```bash
sudo systemctl status degirum.service
```

{% endstep %}

{% step %}
&#x20;**Enable the service on startup**

Use `ystemctl` to automatically enable the degirum service on startup:

{% code overflow="wrap" %}

```bash
sudo systemctl enable degirum.service
```

{% endcode %}
{% endstep %}
{% endstepper %}

### Docker Container

To run the AI server as a Docker container, follow these steps:

{% stepper %}
{% step %}
**Ensure Docker is installed**

Refer to the [official Docker documentation](https://docs.docker.com/engine/install/) for installation instruction
{% endstep %}

{% step %}
**Prepare the local model zoo**

If hosting models locally, create and populate the model zoo directory as described in Starting AI Server from Terminal.
{% endstep %}

{% step %}
**Run the Docker container**

For hosting models locally:

{% code overflow="wrap" %}

```bash
docker run --name <your_server_name> -d -p 8778:8778 -v <your local model zoo>:/zoo --privileged degirum/<your_server_name>:<version>
   
```

{% endcode %}

For AI Hub-only hosting:

{% code overflow="wrap" %}

```bash
docker run --name <your_server_name> -d -p 8778:8778 --privileged degirum/<your_server_name>:<version>
```

{% endcode %}
{% endstep %}
{% endstepper %}

## Rescanning Model Zoos

If you started your AI server in a terminal or as a Linux service, you can tell the AI server to rescan the local model zoo directory by executing the following command on the same host: `degirum server rescan-zoo`

If you started your AI server in the Docker container, then you should rescan the model zoo directory by restarting the container: `docker restart <your_server_name>`