> ## Documentation Index
> Fetch the complete documentation index at: https://docs.condense.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Build your First Pipeline
A **Pipeline** in Condense is the visual and functional representation of data flow between deployed connectors, transforms, and utilities inside a workspace. It is automatically materialized when a connector or transform is deployed. No manual creation needed.
**Before you begin**, make sure you have:
* A workspace created and configured
* At least one environment linked to the workspace
* Workspace Admin or Maintainer role
## What You'll Build
```text theme={null}
[ Input Connector ] → [ Kafka Topic ] → [ Transform ] → [ Output Connector ]
(Data Source) (Stream) (Process) (Destination)
```
By the end of this guide, you will have a live pipeline ingesting data from a source, processing it through a transform, and delivering it to a destination, all inside your Condense workspace.
***
## Step 1 - Deploy an Input Connector
An **Input Connector** receives data from an external system and injects it into your pipeline.
1. Navigate to your **Workspace** → click **Connectors** in the left sidebar
2. Click **+ Add Connector**
3. Select **Input** as the connector role
4. Choose your connector type (e.g. Kafka, HTTP, Teltonika, Pub/Sub)
5. Fill in the required configuration fields:
* **Title** - a unique name for this connector
* **Output Topic** - the Kafka topic this connector will publish data to
* **Environment Variables** - credentials, endpoints, or protocol settings
6. Click **Deploy**
Once deployed, your Input Connector will appear as a block on the Pipeline canvas and begin publishing data to the configured output topic.
***
## Step 2 - Add a Transform
A **Transform** processes messages as they flow between your Input and Output connectors.
#### Option A - Prebuilt Transform
1. On the **Pipeline canvas**, click **+ Add Transform**
2. Select **Prebuilt**
3. Choose a transform type:
* **Geofence** - location-based event triggering
* **Alert Utility** - trigger alerts when a condition is met
* **Split Utility** - route messages based on rules
4. Configure the required parameters:
* **Input Topic** - must match the Output Topic from your Input Connector
* **Output Topic** - the topic the transform will publish results to
5. Click **Deploy**
#### Option B - Custom Transform
1. Go to **Applications** → create or select an existing application
2. Build, test, and publish the application
3. Back on the Pipeline canvas, click **+ Add Transform** → **Custom**
4. Select your published application and version
5. Configure input/output topics and environment variables
6. Click **Deploy**
***
## Step 3 - Deploy an Output Connector
An **Output Connector** takes the processed data and delivers it to an external destination.
1. Navigate to **Connectors** → click **+ Add Connector**
2. Select **Output** as the connector role
3. Choose your connector type (e.g. PostgreSQL, Big Query, S3, Kafka)
4. Fill in the required configuration fields:
* **Title** - a unique name for this connector
* **Input Topic** - must match the Output Topic from your Transform
* **Environment Variables** - destination credentials and settings
5. Click **Deploy**
Condense automatically draws a connecting line between blocks that share matching topics. Your pipeline is now live.
***
## Step 4 - View Your Pipeline
Click **Pipeline** in the left sidebar. You will see all deployed blocks connected by lines:
Clicking any block opens its **Component Detail Panel** showing the connector name, category, type, status, and Kafka topic information:
Pipelines are automatically materialized. They appear automatically as soon as connectors and transforms are deployed. You do not manually create a pipeline.
***
## Step 5 - Monitor Your Pipeline
#### Real-Time Metrics
Each block displays live CPU and memory utilization:
Colour-coded status indicators let you instantly identify Running, Stopped, or Error states:
#### Logs and Debugging
Access detailed real-time logs for each block with customizable streaming intervals (5 seconds by default):
### Editing Your Pipeline
If you need to fix a misconfiguration after deployment, use **Edit Configurations**:
1. Click the block you want to edit on the Pipeline canvas
2. Click **Edit Configuration**
3. Update the required fields
4. Click **Save Changes**
Fields with a **lock icon** cannot be edited after deployment. To change locked fields, delete and redeploy the component.
### Roles and Permissions
| **Operation** | **Admin** | **Maintainer** | **Developer** | **Viewer** |
| ----------------------------------- | :-------: | :------------: | :-----------: | :--------: |
| Deploy pre-built connectors | **✓** | **✓** | **✕** | **✕** |
| Deploy custom connectors/transforms | **✓** | **✓** | **✕** | **✕** |
| Configure deployed components | **✓** | **✓** | **✕** | **✕** |
| Delete deployed components | **✓** | **✓** | **✕** | **✕** |
| View pipelines and connections | **✓** | **✓** | **✓** | **✓** |
| View component logs/configuration | **✓** | **✓** | **✓** | **✓** |
***
## Best Practices
**✓** Use **clear topic naming** to make the pipeline canvas self-explanatory
**✓ Group related connectors and transforms** logically to simplify understanding
**✓ Avoid unused topic links** to reduce visual clutter
**✓ Regularly check component logs** to catch issues early
**✓** Document the pipeline purpose in the workspace description
***
## Common Pitfalls
* **Misaligned Topic Names** - Double-check that Output Topic of one block exactly matches the Input Topic of the next block
* **Unused Components Left Running** - Remove or disable unused connectors to prevent unnecessary resource usage
* **Overcrowded Canvas** - Use multiple workspaces when a single pipeline becomes too dense