> For the complete documentation index, see [llms.txt](https://docs.sec1.io/user-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sec1.io/user-docs/10-siem-edr-xdr/data-collection.md).

# Data Collection

There are two ways to get data into Sec1 SIEM + EDR/XDR: run the **Sec1 agent**, or **connect an existing log collector** you already operate. You can mix both — an agent on your endpoints and servers plus your existing syslog or Logstash pipeline feeding the same platform. This page explains each path and how the single agent covers SIEM, EDR and XDR at once.

## Two ways to get data in

```mermaid
flowchart LR
    subgraph Env["Your environment"]
        H["Endpoints & servers"]
        S["Existing shippers<br/>(Logstash / Beats / rsyslog)"]
    end
    A["Sec1 agent"]
    I["Sec1 XDR Platform<br/>(ingest, token-authenticated)"]

    H -->|telemetry + logs| A
    S -->|to agent receiver| A
    S -->|direct to platform + token| I
    A -->|outbound 443| I
```

* **Use our agent** — one unified binary tails log files, receives syslog, reads Windows Event Log channels and (for EDR/XDR) collects endpoint telemetry. Install it once per host; the platform pushes the rest.
* **Connect an existing collector** — keep your current shipper and either point it at an agent receiver or push straight to the platform ingest with a token. No agent required on the log source itself.

## Use the Sec1 agent

The agent is a single static binary you install per host. What it collects is configured centrally in the console and pushed down — you don't hand-edit files on each host. It can do all of the following at once:

* **Tail log files** — watch paths on the host and forward new lines.
* **Receive syslog** — listen for RFC 3164 / RFC 5424 syslog over UDP or TCP.
* **Read Windows Event Log** — tail channels such as Security, System, Application and Sysmon.
* **Run log receivers** — accept pushes from existing shippers over HTTP or raw TCP (see below).
* **Collect endpoint telemetry (EDR/XDR)** — processes, network connections, file integrity, installed software and connected peripherals.

Configure inputs in the console under the agent's **Agent Inputs** editor — files, syslog, Windows Event Log channels and receivers are written to the managed config and applied to the agent without a reinstall. See [Install the agent](/user-docs/10-siem-edr-xdr/install-the-agent.md).

## Connect an existing log collector

Already running Logstash, Beats, rsyslog or a custom shipper? Keep it. You have two ways to bring that data into the platform.

### Agent-receiver mode

Point your existing shipper at a **receiver** on a Sec1 agent. The agent normalises, batches and forwards to the platform, and buffers to its on-disk spool during an outage — so your pipeline inherits guaranteed delivery without changing your shipper's own logic.

Receivers are **off by default**. You turn one on centrally — nothing to configure on the host:

1. In the console, open **Agent Inputs** (Data collection → the agent's inputs editor).
2. Select the host (or a group) and **Add input → Receiver**.
3. Choose the receiver type, set the bind port and source type, and **Save**:
   * **HTTP receiver** — accepts HTTP POSTs (JSON or newline-delimited). Optionally require a bearer token. Default bind `:9099`.
   * **Raw TCP receiver** — accepts newline-delimited events over a TCP socket. Default bind `:9100`.
   * **Syslog** — accepts RFC 3164 / RFC 5424 over UDP or TCP. Default bind `:5514`.
4. The agent picks up the change on its next check-in (seconds) and starts listening — **no reinstall**.
5. Open that port **only on the internal network** between your shippers and the agent (see the allowlist in [Install the agent](/user-docs/10-siem-edr-xdr/install-the-agent.md)), then point your shipper at `agent-host:<port>`.

Each receiver is tagged with a source type so the platform parses it to the right schema.

#### Logstash example

Point a Logstash `http` output at the agent's HTTP receiver. The receiver accepts a single JSON object, a JSON array (each element becomes one event) or newline-delimited JSON. Include the `Authorization` header only if you set a bearer token on the receiver:

```ruby
output {
  http {
    url         => "http://AGENT-HOST:9099/ingest"
    http_method => "post"
    format      => "json"
    headers     => { "Authorization" => "Bearer <receiver-token>" }   # only if a token is set on the receiver
  }
}
```

Enable the HTTP receiver in the console first (**Agent Inputs → Add input → Receiver**), and open `:9099` only on the internal network between Logstash and the agent.

#### Filebeat / Beats

Filebeat and other Beats can ship to the agent two ways:

* **`output.http`** (Elastic 8.x+) — POST to the agent HTTP receiver at `http://AGENT-HOST:9099/ingest` with an optional `Authorization: Bearer <receiver-token>` header.
* **`output.logstash`** — send to a Logstash instance and let Logstash forward to the agent using the `http` output block above.

#### rsyslog

Forward from rsyslog to the agent's **syslog** receiver (`:5514`) with `omfwd`. TCP (`@@`) is shown; use a single `@` for UDP:

```
*.* action(type="omfwd" target="AGENT-HOST" port="5514" protocol="tcp"
           template="RSYSLOG_SyslogProtocol23Format")
```

Enable the syslog receiver in the console and open `:5514` only on the internal network.

### Direct-to-platform mode

Point your shipper straight at the **Sec1 XDR Platform** ingest and authenticate with a token — no agent in the path. This suits centralised pipelines that already aggregate everything in one place. The ingest endpoint is token-authenticated and reached outbound over HTTPS on the platform gateway; every event is attributed to the tenant the token belongs to.

#### Logstash example

Post straight to the platform ingest at `https://api.sec1.io/siem/ingest` with your **ingest token**. Use the `json_batch` format so Logstash sends events as a JSON array (the shape the ingest endpoint expects):

```ruby
output {
  http {
    url         => "https://api.sec1.io/siem/ingest"
    http_method => "post"
    format      => "json_batch"
    headers     => { "Authorization" => "Bearer <ingest-token>" }
  }
}
```

Get the ingest token from the console. No agent or open port is needed for this path — Logstash reaches the platform outbound over HTTPS (443).

Use agent-receiver mode when you want the agent's on-disk spool and local batching in front of the platform; use direct-to-platform when your pipeline is already centralised and you'd rather not add a hop. Either way the agent is **optional for log collection (SIEM)** — but EDR/XDR endpoint telemetry and response require the agent on the host.

## One agent for SIEM, EDR, XDR or all of it

There is **one** agent binary. You do not pick a SIEM build or an EDR build — the same agent does whatever it is entitled and enabled to do. What runs on a given host is the intersection of two things:

**Effective capability = licensed (subscription) AND toggled-on (per host).**

* **Licensed** — your subscription entitles SIEM, EDR and/or XDR. The platform pushes this entitlement to the agent. With no active subscription the agent goes dormant.
* **Toggled-on** — an operator turns SIEM / EDR / XDR on or off **per host** in the console, sets a **fleet default** or hints the initial choice at install with `SEC1_CAPABILITIES=siem,edr`. This lets you license a product broadly but enable it only where you want it.

A capability runs only when it is **both** licensed and toggled on. Change either side in the console and the agent adjusts on its next config poll — no reinstall.

### Kernel-grade telemetry

Where the OS allows it, the EDR/XDR agent captures at the kernel rather than inferring from polling:

| OS       | Method                      | Notes                                                                |
| -------- | --------------------------- | -------------------------------------------------------------------- |
| Linux    | eBPF                        | Process and network events captured in-kernel.                       |
| Windows  | ETW + Windows Event Log     | Event tracing plus Security/System/Application/Sysmon channels.      |
| macOS    | Endpoint Security framework | Requires the Apple entitlement; visibility-focused.                  |
| Fallback | Polling                     | Used where a kernel source isn't available, so a host still reports. |

The agent reports its active capture method (for example `ebpf`, `etw-sysmon` or `poll`) in its heartbeat, so you can see the fidelity each host is running at.

## Where to go next

* [**Install the agent**](/user-docs/10-siem-edr-xdr/install-the-agent.md) — prerequisites, the allowlist and single vs bulk install.
* [**Architecture**](/user-docs/10-siem-edr-xdr/architecture.md) — the end-to-end picture from ingest to response.
* [**Endpoints & telemetry**](/user-docs/10-siem-edr-xdr/endpoints-and-telemetry.md) — what each host reports.