> For the complete documentation index, see [llms.txt](https://data-programs.gitbook.io/singularity/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://data-programs.gitbook.io/singularity/data-preparation/get-started.md).

# Get Started

Follow these steps to set up and start using Singularity.

## 1. Initialize the Database

If you're using Singularity for the first time, you'll need to initialize the database. This step is required only once.

```sh
singularity admin init
```

## 2. Connect to Storage Systems

Singularity partners with RClone to provide seamless integration with over 40 different storage systems. These storage systems can play two main roles:

* **Source Storage**: This is where the dataset is currently stored and where Singularity will source data from for preparation.
* **Output Storage**: This is the destination where Singularity will store the CAR (Content Addressable Archive) files after processing. Choose a storage system appropriate for your needs and connect it with Singularity to start preparing your datasets.

### 2a. Add a local file system

The most command storage system is the local file system. To add a folder as a source storage to singularity:

```sh
singularity storage create local --name "my-source" --path "/mnt/dataset/folder"
```

### 2b. Add a S3 data source

Any S3 compatible storage system can be used, including AWS S3, MinIO, etc. Below is an example for public dataset

```sh
singularity storage create s3 aws --name "my-source" --path "public-dataset-test"
```

## 3. Create a preparation

```sh
singularity prep create --source "my-source" --name "my-prep"
```

## 4. Run the preparation worker

```sh
singularity prep start-scan my-prep my-source
singularity run dataset-worker
```

## 5. Check the preparation status and result

```sh
singularity prep status my-prep
singularity prep list-pieces my-prep
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://data-programs.gitbook.io/singularity/data-preparation/get-started.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.