Development Setup

This guide covers setting up a local development environment for contributing to Logstyx or customizing it for your needs.

Development Workflow

Docker Development (Recommended)

Run all services in Docker for the complete development environment:

bash

# Start all services
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d

After making code changes, rebuild the specific service:

bash

# Rebuild and restart a specific service
docker compose up -d --build gateway-service

# Or rebuild all services at once
docker compose up -d --build

Quick Rebuilds

Docker caches unchanged layers, so rebuilds are relatively fast. Work on one service at a time for the quickest iteration cycle.

Alternative: Full Local Development

For the fastest development experience with hot-reload and debugging capabilities, you can run all services locally on your host machine.

Quick Start with dev.sh

The easiest way to set up local development:

bash

# Make the script executable
chmod +x dev.sh dev-stop.sh

# Start everything
./dev.sh

The script will:

✅ Run the installer if .env.encrypted doesn't exist
✅ Start external dependencies (MongoDB, Redis, RabbitMQ) in Docker
✅ Configure WebSocket hosts for local development
✅ Distribute .env to all microservices
✅ Start all services (auth, cache, core, utility, gateway, frontend)
✅ Open everything in tmux (if available) or run in background

With tmux (recommended):

All services run in split panes
Easy to view logs and monitor
Attach/detach with Ctrl+B then D
Navigate panes with Ctrl+B then arrow keys

Without tmux:

Services run in background
Logs saved to .logs/ directory
View logs: tail -f .logs/*.log

To stop everything:

bash

./dev-stop.sh

Manual Setup (Advanced)

If you prefer to set up manually or want to understand what dev.sh does:

Click to expand manual instructions

Step 1: Generate encrypted configuration

bash

./install.sh

When prompted "Start Logstyx now?", choose No (we'll run locally instead).

Step 2: Start external dependencies

bash

docker compose up -d mongodb redis rabbitmq

Step 3: Configure local WebSocket hosts

Edit .env.encrypted and add these lines:

bash

# Local Development WebSocket Hosts
AUTH_WSHOST=ws://localhost:5001/rpc
CACHE_WSHOST=ws://localhost:5002/rpc
UTILITY_WSHOST=ws://localhost:5003/rpc
CORE_WSHOST=ws://localhost:5004/rpc

Step 4: Distribute .env to microservices

bash

# Copy to each service's src directory
cp .env.encrypted microservice/auth-service/src/.env
cp .env.encrypted microservice/cache-service/src/.env
cp .env.encrypted microservice/core-service/src/.env
cp .env.encrypted microservice/utility-service/src/.env
cp .env.encrypted microservice/frontend/src/.env

Step 5: Start all services in separate terminals

bash

# Terminal 1: Auth Service (port 5001)
cd microservice/auth-service && pnpm start:local

# Terminal 2: Cache Service (port 5002)
cd microservice/cache-service && pnpm start:local

# Terminal 3: Core Service (port 5004)
cd microservice/core-service && pnpm start:local

# Terminal 4: Utility Service (port 5003)
cd microservice/utility-service && pnpm start:local

# Terminal 5: Gateway (port 5000)
node simple-gateway.js --port=5000

# Terminal 6: Frontend
cd microservice/frontend && pnpm dev

Step 6: Access the application

Open your browser at http://localhost:5000

Prerequisites

Before you begin, ensure you have the following installed:

Docker (version 20.10 or higher)
Docker Compose (version 2.0 or higher)
pnpm (version 8.0 or higher)
Node.js (version 18 or higher)
Git

Quick Check

Verify your installations:

bash

docker --version
docker compose version
pnpm --version
node --version

Clone the Repository

Start by cloning the Logstyx repository and its dependencies:

bash

# Clone the main repository
git clone https://github.com/devatlogstyx/logstyx.git
cd logstyx

Install Dependencies

Logstyx uses a monorepo structure managed with pnpm workspaces. Install all dependencies across all microservices:

bash

pnpm install

This will install dependencies for:

Auth Service
Cache Service
Core Service
Gateway Service
Utility Service
Frontend Application

pnpm Required

Logstyx uses pnpm workspaces. Using npm or yarn may cause dependency resolution issues.

Environment Configuration

Logstyx uses encryption for all sensitive configuration values. The only way to properly set up your environment is using the interactive installer.

Run the Interactive Installer

bash

./install.sh

The installer will:

Prompt for your credentials:
- Admin username
- Admin email
- Admin password
- Project title
Generate secure keys:
- Encryption keys (CRYPTO_SECRET)
- JWT secrets (REFRESH_TOKEN_SECRET, USER_AUTHENTICATION_JWT_SECRET)
- Master key (MASTER_KEY)
Encrypt sensitive values:
- All credentials are encrypted with ENC_ prefix
- Only MASTER_KEY remains in plain text (used for decryption)
Create your .env file:
- Ready to use immediately
- All sensitive data protected

Security First

Do NOT manually create a .env file with plain text values. The installer ensures all sensitive configuration is properly encrypted. Manual configuration would expose your credentials and defeat Logstyx's security model.

Using External Services (Optional)

If you want to use external MongoDB, Redis, or RabbitMQ instead of the bundled Docker services, you can add these during or after installation.

The installer will ask if you want to configure external services. If you choose to add them later, edit your .env and add:

bash

# External service URLs (optional)
# Leave these commented to use bundled Docker services
AMQP_HOST=amqp://user:password@your-rabbitmq:5672
REDIS_URL=redis://your-redis:6379
MONGODB_HOST=mongodb://your-mongo:27017/logstyx

Default Behavior

By default, Logstyx uses bundled services in Docker. You don't need to configure external services for development.

Start Development Services

Logstyx provides a development-optimized Docker Compose configuration:

bash

# Start all services in development mode
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d

This will start:

MongoDB - Database (port 27017)
Redis - Cache (port 6379)
RabbitMQ - Message queue (port 5672, management UI on 15672)
Auth Service - Authentication (port 5001)
Cache Service - Caching layer (port 5002)
Core Service - Core processing (port 5004)
Gateway Service - API gateway (port 5000)
Utility Service - Utilities (port 5003)
Frontend - Web UI (port 5000)

Verify Services

Check that all services are running:

bash

docker compose ps

View logs for all services:

bash

docker compose logs -f

View logs for a specific service:

bash

docker compose logs -f gateway-service

Access the Application

Once all services are running:

Frontend Application: http://localhost:5000
API Gateway: http://localhost:3000/api/*
RabbitMQ Management: http://localhost:15672 (guest/guest)

Microservices Structure

microservice/
├── auth-service/         # User authentication & authorization
├── cache-service/        # Caching layer (Redis wrapper)
├── core-service/         # Core log processing
├── gateway-service/      # API gateway & routing
├── utility-service/      # Utility functions
├── frontend/            # React + Vite web interface
└── common/              # Shared code & utilities

Each service can be developed independently while maintaining communication through the Docker network.

Working with SDKs

During development, you may want to test the SDKs locally:

Clone SDK Repositories

bash

# Create a workspace directory
mkdir -p ../logstyx-sdks
cd ../logstyx-sdks

# Clone SDK repositories
git clone https://github.com/devatlogstyx/logstyx-js-core.git
git clone https://github.com/devatlogstyx/logstyx-js-node.git
git clone https://github.com/devatlogstyx/logstyx-js-browser.git
git clone https://github.com/devatlogstyx/logstyx-js-express.git
git clone https://github.com/devatlogstyx/logstyx-js-react.git
git clone https://github.com/devatlogstyx/logstyx-js-react-native.git

Test SDK Integration

Create a test application:

bash

cd ../
mkdir test-app
cd test-app
npm init -y
npm install ../logstyx-sdks/logstyx-js-node

Create test.js:

javascript

const { LogstyxNode } = require('logstyx-js-node');

const logger = new LogstyxNode({
  apiKey: 'your-api-key',
  endpoint: 'http://localhost:3000'
});

logger.info('Testing Logstyx SDK', {
  environment: 'development',
  service: 'test-app'
});

Run the test:

bash

node test.js

Inspect Service Internals

Access a running container's shell:

bash

docker compose exec gateway-service sh

Database Access

Connect to MongoDB:

bash

docker compose exec mongodb mongosh CoreService

Connect to Redis CLI:

bash

docker compose exec redis redis-cli

Common Development Tasks

Update Dependencies

Update all dependencies across the monorepo:

bash

pnpm update -r

Add New Dependencies

Add a dependency to a specific service:

bash

cd microservice/core-service
pnpm add new-dependency

Troubleshooting

Ports Already in Use

If you get port conflicts:

bash

# Check what's using a port
lsof -i :5000

# Kill the process
kill -9 <PID>

Or modify the ports in docker-compose.dev.yml.

Services Won't Start

Check Docker resources:

bash

docker system df
docker system prune  # Clean up unused resources

Ensure you have enough:

RAM: 4GB minimum
Disk: 10GB free space

RabbitMQ Connection Issues

Restart RabbitMQ:

bash

docker compose restart rabbitmq

Check RabbitMQ is healthy:

bash

docker compose exec rabbitmq rabbitmq-diagnostics ping

MongoDB Connection Errors

Check MongoDB logs:

bash

docker compose logs mongodb

Verify connection string in .env matches the container name.

Next Steps

Now that you have Logstyx running locally:

Explore the Architecture - Understand how services communicate
Try the SDKs - Integrate Logstyx into your applications

Getting Help

If you encounter issues:

GitHub Issues: Report bugs or request features
Discussions: Ask questions and share ideas
Documentation: Check other sections of this guide for more details

Development Setup ​

Development Workflow ​

Docker Development (Recommended) ​

Alternative: Full Local Development ​

Quick Start with dev.sh ​

Manual Setup (Advanced) ​

Prerequisites ​

Clone the Repository ​

Install Dependencies ​

Environment Configuration ​

Run the Interactive Installer ​

Using External Services (Optional) ​

Start Development Services ​

Verify Services ​

Access the Application ​

Microservices Structure ​

Working with SDKs ​

Clone SDK Repositories ​

Test SDK Integration ​

Inspect Service Internals ​

Database Access ​

Common Development Tasks ​

Update Dependencies ​

Add New Dependencies ​

Troubleshooting ​

Ports Already in Use ​

Services Won't Start ​

RabbitMQ Connection Issues ​

MongoDB Connection Errors ​

Next Steps ​

Getting Help ​

Development Setup

Development Workflow

Docker Development (Recommended)

Alternative: Full Local Development

Quick Start with dev.sh

Manual Setup (Advanced)

Prerequisites

Clone the Repository

Install Dependencies

Environment Configuration

Run the Interactive Installer

Using External Services (Optional)

Start Development Services

Verify Services

Access the Application

Microservices Structure

Working with SDKs

Clone SDK Repositories

Test SDK Integration

Inspect Service Internals

Database Access

Common Development Tasks

Update Dependencies

Add New Dependencies

Troubleshooting

Ports Already in Use

Services Won't Start

RabbitMQ Connection Issues

MongoDB Connection Errors

Next Steps

Getting Help