Integration with Logstyx

This guide shows how to integrate Probestyx with Logstyx to send metrics to your centralized logging platform.

Overview

Probestyx collects metrics and exposes them via HTTP. Logstyx probes periodically fetch these metrics and store them as logs in your Logstyx instance.

┌──────────────┐      HTTP GET        ┌────────────────┐      Store      ┌──────────┐
│  Probestyx   │ ◄──────────────────  │ Logstyx Probe  │ ──────────────► │ Logstyx  │
│   Agent      │   /metrics           │                │     as logs     │ Platform │
│  :9100       │  (every 60s)         │                │                 │          │
└──────────────┘                       └────────────────┘                 └──────────┘

Prerequisites

Before integrating:

Probestyx installed and running on your servers
- See Installation
Logstyx instance running and accessible
- See Logstyx documentation
Network connectivity between Logstyx and your Probestyx agents
Logstyx project created to receive metrics

Integration Steps

Step 1: Configure Probestyx

Create a config.yaml with the metrics you want to collect:

yaml

server:
  port: 9100
  secret: ""  # Optional: add secret for authentication

system:
  enabled: true
  metrics:
    # Essential system metrics
    - cpu_usage_percent
    - ram_usage_percent
    - disk_usage_percent
    - network_bytes_sent_per_sec
    - network_bytes_recv_per_sec
    - system_uptime_seconds
    - hostname

scrapers:
  # Add custom metrics if needed
  - name: app_metrics
    source:
      type: url
      url: "http://localhost:8080/metrics"
      format: json
    metrics:
      - path: "requests.total"
        name: "app_requests_total"

Step 2: Test Probestyx Endpoint

Verify Probestyx is exposing metrics:

bash

# Test the endpoint
curl http://localhost:9100/metrics

# Expected output: JSON with metrics
{
  "cpu_usage_percent": 45.2,
  "ram_usage_percent": 67.8,
  "disk_usage_percent": 52.3,
  "network_bytes_sent_per_sec": 102400,
  "network_bytes_recv_per_sec": 204800,
  "system_uptime_seconds": 86400,
  "hostname": "web-server-01",
  "app_requests_total": 150000
}

Step 3: Create Logstyx Probe

In the Logstyx dashboard:

Navigate to Probes → Add Probe
Basic Configuration:
- Title: Descriptive name (e.g., "Production Web Server 01")
- Project: Select your project
- Interval: 60 seconds (recommended)
- Context (optional): Add server identification
  json
```
{
  "server": "web-01",
  "environment": "production",
  "datacenter": "us-east-1"
}
```
  1
  2
  3
  4
  5
Connection Settings:
- URL: http://your-server:9100/metrics
  - Replace your-server with actual hostname/IP
  - Examples:
    - http://192.168.1.100:9100/metrics
    - http://web-server-01.internal:9100/metrics
    - http://10.0.1.50:9100/metrics
- HTTP Method: GET
- Timeout: 10000 ms (10 seconds)
Authentication:
- If Probestyx has no secret:
  - Type: None
- If Probestyx has a secret:
  - Type: Probestyx
  - Probestyx Secret: Enter the same secret from your Probestyx config
Test Connection:
- Click Test Connection
- Should show: "Connected successfully"
- If it fails, see Troubleshooting
Create:
- Click Create to start collecting metrics

Step 4: Verify Integration

After the first interval (60 seconds):

Go to your project's Logs tab in Logstyx
You should see log entries containing metrics
Each log entry will have:
- Timestamp
- All metrics as JSON
- Context (if provided)

Example log entry:

json

{
  "timestamp": "2024-01-01T12:00:00Z",
  "context": {
    "server": "web-01",
    "environment": "production"
  },
  "data": {
    "cpu_usage_percent": 45.2,
    "ram_usage_percent": 67.8,
    "disk_usage_percent": 52.3,
    "network_bytes_sent_per_sec": 102400,
    "network_bytes_recv_per_sec": 204800,
    "system_uptime_seconds": 86400,
    "hostname": "web-server-01",
    "app_requests_total": 150000
  }
}

Authentication Setup

Without Authentication (Simple)

Probestyx config:

yaml

server:
  port: 9100
  # No secret = no authentication

Logstyx probe:

Authentication Type: None

With Authentication (Recommended)

Probestyx config:

yaml

server:
  port: 9100
  secret: "your-secure-secret-key-here"

Logstyx probe:

Authentication Type: Probestyx
Probestyx Secret: your-secure-secret-key-here

WARNING

Keep your secret secure and never commit it to version control. The secret must match exactly on both Probestyx and Logstyx.

Multi-Server Setup

To monitor multiple servers:

Option 1: One Probe per Server

Create a separate probe for each server:

Server 1 - web-01:

Title: Production Web Server 01
URL: http://web-01.internal:9100/metrics
Context: {"server": "web-01", "role": "web"}

Server 2 - web-02:

Title: Production Web Server 02
URL: http://web-02.internal:9100/metrics
Context: {"server": "web-02", "role": "web"}

Server 3 - db-01:

Title: Production Database Server
URL: http://db-01.internal:9100/metrics
Context: {"server": "db-01", "role": "database"}

Option 2: Grouped by Environment

Create different projects for different environments:

Production Project:

Probe: Production Web 01 → web-01:9100
Probe: Production Web 02 → web-02:9100
Probe: Production DB → db-01:9100

Staging Project:

Probe: Staging Web → staging-web:9100
Probe: Staging DB → staging-db:9100

Network Configuration

Firewall Rules

Ensure Logstyx can reach Probestyx:

On Probestyx servers:

bash

# Allow inbound on port 9100 from Logstyx server
# UFW example:
sudo ufw allow from <logstyx-ip> to any port 9100

# iptables example:
sudo iptables -A INPUT -p tcp -s <logstyx-ip> --dport 9100 -j ACCEPT

Docker Networks

If running in Docker:

Probestyx in Docker:

yaml

# docker-compose.yml
services:
  probestyx:
    image: devatlogstyx/probestyx:latest
    ports:
      - "9100:9100"
    volumes:
      - ./config.yaml:/root/config.yaml:ro

Same Docker network as Logstyx:

yaml

# docker-compose.yml
services:
  probestyx:
    image: devatlogstyx/probestyx:latest
    networks:
      - logstyx-network
    volumes:
      - ./config.yaml:/root/config.yaml:ro

networks:
  logstyx-network:
    external: true

Then use service name in URL: http://probestyx:9100/metrics

Cloud Environments

AWS:

Use private IPs or DNS names
Configure security groups to allow port 9100
Consider VPC peering if across VPCs

GCP:

Use internal DNS
Configure firewall rules
Use VPC networks

Azure:

Use internal IPs
Configure Network Security Groups (NSG)
Use VNet peering if needed

Querying Metrics in Logstyx

Once integrated, query your metrics:

Filter by Server

context.server = "web-01"

Filter by Metric Value

data.cpu_usage_percent > 80

Combine Filters

context.environment = "production" AND data.disk_usage_percent > 90

Time-Based Queries

Logstyx automatically timestamps all logs, allowing you to:

View metrics over time
Compare historical data
Identify trends
Set up alerts

Best Practices

1. Use Meaningful Context

Add context to identify sources:

json

{
  "server": "web-01",
  "environment": "production",
  "datacenter": "us-east-1",
  "role": "web",
  "team": "platform"
}

2. Set Appropriate Intervals

yaml

# Real-time monitoring
interval: 10  # 10 seconds

# Standard monitoring
interval: 60  # 1 minute (recommended)

# Low-frequency monitoring
interval: 300  # 5 minutes

3. Group Probes Logically

By Environment:

Production project
Staging project
Development project

By Service:

Web servers project
Database servers project
Cache servers project

By Region:

US-East project
EU-West project
Asia-Pacific project

4. Use Consistent Naming

# Good
Title: "Production Web Server 01 - us-east-1"
Title: "Production Database Master - us-east-1"
Title: "Staging Web Server - us-west-2"

# Avoid
Title: "Server 1"
Title: "DB"
Title: "Test"

5. Monitor Integration Health

Check probe status regularly in Logstyx dashboard
Set up alerts for failed probes
Monitor probe execution time
Review error logs

Advanced Integration

Custom Transformations

If you need to transform data before sending to Logstyx, do it in Probestyx:

yaml

scrapers:
  - name: app_metrics
    source:
      type: url
      url: "http://localhost:8080/metrics"
      format: json
    metrics:
      # Transform in Probestyx
      - path: "response.time"
        name: "response_time_ms"
        calculate: "value * 1000"
      
      - path: "cache.hit_rate"
        name: "cache_hit_percent"
        calculate: "value * 100"

Selective Metric Collection

Only send necessary metrics to reduce storage:

yaml

system:
  enabled: true
  metrics:
    # Only essential metrics
    - cpu_usage_percent
    - ram_usage_percent
    - disk_usage_percent

Aggregated Metrics

Combine multiple sources in Probestyx before sending:

yaml

system:
  enabled: true
  metrics:
    - cpu_usage_percent
    - ram_usage_percent

scrapers:
  - name: app_metrics
    source:
      type: url
      url: "http://localhost:8080/metrics"
      format: json
    metrics:
      - path: "requests.total"
        name: "app_requests_total"
  
  - name: db_metrics
    source:
      type: url
      url: "http://localhost:5432/metrics"
      format: json
    metrics:
      - path: "connections.active"
        name: "db_connections_active"

All metrics will be in a single log entry in Logstyx.

Error Log Monitoring

Beyond numeric metrics, Probestyx can surface actual error lines from nginx or docker logs via format: log, so a Logstyx probe's log entries include the error text itself, not just a count:

yaml

scrapers:
  - name: nginx_errors
    source:
      type: file
      path: "/var/log/nginx/error.log"
      format: log
      max_lines: 50
    metrics:
      - match: "count"
        name: "nginx_error_count"
      - match: "lines"
        name: "nginx_error_lines"

Each scrape only reports lines that are new since the last one — see Scrapers → Log Parser for how that tailing works.

This pattern — one Logstyx probe polling one Probestyx instance on a fixed interval — is exactly the single-consumer case Probestyx defaults to, so no extra configuration is needed. server.consumers + ?callerId= only matter if something else besides your Logstyx probe also polls the same instance's /metrics endpoint.

Troubleshooting

Connection Test Fails

Error: "Connection refused"

Verify Probestyx is running: curl http://server:9100/health
Check firewall allows port 9100
Verify network connectivity: ping server-ip

Error: "Timeout"

Increase timeout in Logstyx probe settings
Check network latency
Verify Probestyx is responding: curl -w "@curl-time" http://server:9100/metrics

Error: "Not found"

Verify URL is correct
Check endpoint exists: curl http://server:9100/metrics
Ensure path is /metrics

Authentication Errors

Error: "Authentication failed"

Verify secret matches on both sides
Check for extra spaces in secret
Ensure secret is set in Probestyx config
Verify Authentication Type is set to "Probestyx" in Logstyx

No Logs Appearing

Probe shows "Success" but no logs:

Wait for first interval to pass
Check you're looking in the correct project
Verify probe is enabled
Check Logstyx logs for errors

Empty log entries:

Verify Probestyx is returning data: curl http://server:9100/metrics
Check Probestyx logs for errors
Ensure metrics are configured in config.yaml

Performance Issues

Slow response times:

Reduce number of scrapers in Probestyx
Scrapers aren't cached (cache_ttl only applies to system metrics) — increase the Logstyx probe's polling interval if a scraper's source is slow
Use filtering to limit metrics
Check source endpoint performance

High memory usage:

Reduce collection frequency (increase interval)
Limit number of metrics collected
Check for memory leaks in scraped sources

Metric Discrepancies

Values seem wrong:

Verify calculations in Probestyx config
Check source data manually
Ensure units are correct
Review Probestyx logs for parsing errors

Migration Guide

From Manual Polling

If you're currently manually querying Probestyx:

Before:

bash

# Manual cron job
*/5 * * * * curl http://server:9100/metrics > /var/log/metrics.json

After:

Create Logstyx probe with 300-second interval
Remove cron job
Query metrics in Logstyx dashboard

From Other Monitoring Systems

From Prometheus:

Keep Prometheus exporter running
Point Probestyx at exporter: format prometheus
Create Logstyx probe pointing at Probestyx
Gradually transition queries to Logstyx

From Telegraf:

Run Probestyx alongside Telegraf
Configure Probestyx to collect same metrics
Create Logstyx probes
Verify data matches
Retire Telegraf when confident

Next Steps

System Metrics - Available system metrics
Custom Metrics - Collect application metrics
Logstyx Documentation - Learn more about Logstyx

Integration with Logstyx ​

Overview ​

Prerequisites ​

Integration Steps ​

Step 1: Configure Probestyx ​

Step 2: Test Probestyx Endpoint ​

Step 3: Create Logstyx Probe ​

Step 4: Verify Integration ​

Authentication Setup ​

Without Authentication (Simple) ​

With Authentication (Recommended) ​

Multi-Server Setup ​

Option 1: One Probe per Server ​

Option 2: Grouped by Environment ​

Network Configuration ​

Firewall Rules ​

Docker Networks ​

Cloud Environments ​

Querying Metrics in Logstyx ​

Filter by Server ​

Filter by Metric Value ​

Combine Filters ​

Time-Based Queries ​

Best Practices ​

1. Use Meaningful Context ​

2. Set Appropriate Intervals ​

3. Group Probes Logically ​

4. Use Consistent Naming ​

5. Monitor Integration Health ​

Advanced Integration ​

Custom Transformations ​

Selective Metric Collection ​

Aggregated Metrics ​

Error Log Monitoring ​

Troubleshooting ​

Connection Test Fails ​

Authentication Errors ​

No Logs Appearing ​

Performance Issues ​

Metric Discrepancies ​

Migration Guide ​

From Manual Polling ​

From Other Monitoring Systems ​

Next Steps ​

Integration with Logstyx

Overview

Prerequisites

Integration Steps

Step 1: Configure Probestyx

Step 2: Test Probestyx Endpoint

Step 3: Create Logstyx Probe

Step 4: Verify Integration

Authentication Setup

Without Authentication (Simple)

With Authentication (Recommended)

Multi-Server Setup

Option 1: One Probe per Server

Option 2: Grouped by Environment

Network Configuration

Firewall Rules

Docker Networks

Cloud Environments

Querying Metrics in Logstyx

Filter by Server

Filter by Metric Value

Combine Filters

Time-Based Queries

Best Practices

1. Use Meaningful Context

2. Set Appropriate Intervals

3. Group Probes Logically

4. Use Consistent Naming

5. Monitor Integration Health

Advanced Integration

Custom Transformations

Selective Metric Collection

Aggregated Metrics

Error Log Monitoring

Troubleshooting

Connection Test Fails

Authentication Errors

No Logs Appearing

Performance Issues

Metric Discrepancies

Migration Guide

From Manual Polling

From Other Monitoring Systems

Next Steps