portfolio

System Architecture - FreelanceOS

Table of Contents

High-Level Overview

FreelanceOS follows an event-driven, microservices-style architecture orchestrated through n8n workflows. Each phase of the freelance lifecycle (discovery, proposals, interviews, delivery, completion) operates as an independent workflow with clear input/output contracts.

Architecture Diagram

graph TB
    subgraph "Phase 1: Discovery"
        A[Email Alerts] --> B[n8n Email Trigger]
        C[Manual Telegram Command] --> B
        B --> D[Job Parser]
        D --> E[Scoring Engine]
        E --> F{Score >= 70?}
        F -->|Yes| G[Qualified Jobs Queue]
        F -->|No| H[Rejected Log]
    end

    subgraph "Phase 2: Proposals"
        G --> I[Context Builder]
        I --> J[Claude API]
        J --> K[Telegram Approval]
        K --> L{Approved?}
        L -->|Yes| M[ClickUp Task Created]
        L -->|No| H
    end

    subgraph "Phase 3: Delivery"
        M --> N[Project Manager]
        N --> O[Toggl Time Tracking]
        N --> P[Daily Reminders]
        N --> Q[Progress Reports]
    end

    subgraph "Phase 4: Completion"
        Q --> R[Completion Checklist]
        R --> S[Retrospective Generator]
        S --> T[Archive & Learnings]
    end

    subgraph "Supporting Infrastructure"
        U[PostgreSQL]
        V[Telegram Bot]
        W[ClickUp API]
    end

    B -.-> U
    I -.-> U
    K -.-> V
    M -.-> W
    N -.-> W
    O -.-> U

    style A fill:#e1f5ff
    style C fill:#e1f5ff
    style J fill:#f0e1ff
    style V fill:#e1ffe1
    style U fill:#ffe1e1

Design Principles

  1. Modularity: Each phase operates independently with well-defined interfaces
  2. Fail-Safe: Every workflow has error handling and fallback to human notification
  3. Mobile-First: All human interactions optimized for Telegram mobile interface
  4. Stateless Workflows: Workflows are stateless by default; state stored in PostgreSQL or staticData
  5. Audit Trail: All decisions logged for analytics and debugging

Core Components

1. Workflow Orchestration Engine (n8n)

Purpose: Central nervous system of FreelanceOS. Orchestrates all automation workflows.

Key Responsibilities:

Architecture Pattern: Event-driven microservices

State Management:

Why n8n over alternatives:

2. AI Engine (Claude API - Sonnet 4.5)

Purpose: Generate high-quality, contextual written content (proposals, interview prep, retrospectives).

Use Cases:

  1. Proposal Generation:
    • Input: Job posting, freelancer profile, past successful proposals
    • Output: Customized 200-400 word proposal
    • Model: claude-sonnet-4-5 (quality over speed)
  2. Interview Preparation:
    • Input: Job posting, submitted proposal, client background
    • Output: Expected questions, suggested answers, negotiation strategy
    • Model: claude-sonnet-4-5
  3. Project Retrospectives:
    • Input: ClickUp completed tasks, Toggl time logs, project notes
    • Output: What went well, what to improve, lessons learned
    • Model: claude-sonnet-4-5

Integration Pattern:

n8n HTTP Request Node
  ├─ POST https://api.anthropic.com/v1/messages
  ├─ Headers: x-api-key, anthropic-version
  ├─ Body: { model, max_tokens, messages }
  └─ Response: content[0].text → Parsed output

Error Handling:

Why Claude over alternatives:

3. Mobile Interface (Telegram Bot)

Purpose: Primary human interface for approvals, notifications, and quick commands.

Core Features:

Inbound (User → System):

Outbound (System → User):

Technical Architecture:

Telegram Servers
      ↓
Single Webhook URL (https://domain.com/webhook/telegram)
      ↓
Master Router Workflow (n8n)
      ↓
   Pattern Matching:
   ├─ /submit_job → Job Discovery Workflow
   ├─ /approve → Proposal Approval Workflow
   ├─ /status → Status Query Workflow
   ├─ callback_query → Button Handler Workflow
   └─ Default → Help Message

Single Webhook Solution:

Message Formatting:

Why Telegram:

4. Project Management (ClickUp)

Purpose: Centralized project tracking, task management, and pipeline visibility.

Data Model:

Workspace: FreelanceOS
  └─ Space: Upwork Pipeline
      ├─ List: Prospects (scored jobs)
      ├─ List: Proposals Sent
      ├─ List: Active Projects
      └─ List: Completed/Archive

Custom Fields:

Statuses:

API Operations:

  1. Create Task: When proposal approved
  2. Update Status: As job progresses through pipeline
  3. Add Comments: Log proposal text, client responses, notes
  4. Get Tasks: Daily digest of active work

Automation Triggers:

Why ClickUp:

5. Time Tracking (Toggl)

Purpose: Automatic time tracking per project for invoicing and profitability analysis.

Integration Pattern:

ClickUp Task Status Change
      ↓
n8n Webhook Listener
      ↓
   IF status = "In Progress"
      ↓
   Toggl API: Start Time Entry
      ├─ Description: ClickUp Task #{id}
      ├─ Project: Mapped from ClickUp
      └─ Tags: [Client name, Task type]

   IF status = "Complete"
      ↓
   Toggl API: Stop Time Entry
      ↓
   Export time summary → Invoice data

Automation Rules:

Data Extracted:

Why Toggl:

Data Flow

End-to-End: Job Discovery → Project Completion

sequenceDiagram
    participant Email as Upwork Email Alert
    participant n8n as n8n Workflows
    participant Claude as Claude API
    participant TG as Telegram
    participant User as User (Mobile)
    participant CU as ClickUp
    participant DB as PostgreSQL

    Email->>n8n: New job alert received
    n8n->>n8n: Parse email, extract job URL
    n8n->>n8n: Fetch full job page (HTTP request)
    n8n->>n8n: Score job (criteria evaluation)

    alt Score >= 70
        n8n->>DB: Log qualified job
        n8n->>Claude: Generate proposal (job + profile context)
        Claude->>n8n: Proposal text
        n8n->>TG: Send to user for approval
        TG->>User: Notification with inline buttons
        User->>TG: Click "Approve"
        TG->>n8n: Callback query (approved)
        n8n->>CU: Create task in "Proposals Sent"
        n8n->>TG: Confirmation message
    else Score < 70
        n8n->>DB: Log rejected job with reason
    end

    Note over User,CU: Time passes, job won

    User->>TG: /start_project <task_id>
    TG->>n8n: Command received
    n8n->>CU: Move task to "Active Projects"
    CU->>n8n: Webhook: Status changed
    n8n->>Toggl: Start time tracking
    n8n->>TG: Project kickoff checklist

    Note over User,Toggl: Project work happens

    User->>TG: /complete_project <task_id>
    TG->>n8n: Command received
    n8n->>Toggl: Stop time tracking, export hours
    n8n->>Claude: Generate retrospective
    Claude->>n8n: Learnings document
    n8n->>CU: Move to "Completed", add retrospective
    n8n->>TG: Completion summary + invoice data

Data Transformation Stages

Stage 1: Email → Structured Job Data

Input: Raw Upwork email HTML
  ↓
Extract: Job title, URL, budget (regex/parsing)
  ↓
Fetch: Full job page via HTTP request
  ↓
Parse: Description, skills, client info (Cheerio/HTML parsing)
  ↓
Output: Structured JSON job object

Stage 2: Job Data → Proposal

Input: Job object + Freelancer profile
  ↓
Build Context: Combine job requirements + relevant experience
  ↓
Claude Prompt: "Generate proposal addressing [key points]"
  ↓
AI Generation: 200-400 word customized proposal
  ↓
Output: Proposal text + metadata

Stage 3: Proposal → ClickUp Task

Input: Approved proposal + job metadata
  ↓
API Call: ClickUp create task
  ├─ Title: Job title
  ├─ Description: Proposal text
  ├─ Custom fields: Budget, URL, score
  └─ Status: "Proposals Sent"
  ↓
Output: ClickUp task ID for tracking

Integration Architecture

API Integration Patterns

Pattern 1: Webhook-Based (Real-Time)

External Service (Telegram, ClickUp)
      ↓
HTTPS POST to n8n webhook URL
      ↓
n8n Webhook Trigger Node
      ↓
Workflow execution begins

Pattern 2: Polling-Based (Scheduled)

n8n Cron Trigger (every 5 minutes)
      ↓
HTTP Request to external API
      ↓
Check for new items
      ↓
Process new items only (de-duplication)

Pattern 3: Request-Response (On-Demand)

Workflow needs external data
      ↓
n8n HTTP Request Node
      ├─ Method: GET/POST
      ├─ Auth: Bearer token / API key
      └─ Error handling: Retry logic
      ↓
Parse response
      ↓
Continue workflow with data

Authentication Management

API Keys Storage:

Rotation Strategy:

State Management

Challenge: Stateless Workflows

n8n workflows are stateless by default. Each execution is independent. This creates challenges for:

Solution: Multi-Tiered State Strategy

Tier 1: workflow.staticData (In-Memory Persistence)

// Built-in n8n feature for cross-execution state
const staticData = workflow.staticData;

// Store pending approvals
staticData.pendingApprovals = staticData.pendingApprovals || {};
staticData.pendingApprovals[jobId] = {
  proposalText: proposal,
  timestamp: Date.now(),
  score: jobScore
};

// Retrieve later in different execution
const pending = staticData.pendingApprovals[jobId];

Use Cases:

Limitations:

Tier 2: PostgreSQL (Long-Term Persistence)

-- Jobs table
CREATE TABLE jobs (
  id UUID PRIMARY KEY,
  upwork_job_id VARCHAR,
  title TEXT,
  description TEXT,
  budget DECIMAL,
  score INTEGER,
  status VARCHAR,
  created_at TIMESTAMP
);

-- Proposals table
CREATE TABLE proposals (
  id UUID PRIMARY KEY,
  job_id UUID REFERENCES jobs(id),
  content TEXT,
  status VARCHAR, -- draft, approved, sent, rejected
  approved_at TIMESTAMP,
  clickup_task_id VARCHAR
);

Use Cases:

Access Pattern:

n8n Postgres Node
  ├─ INSERT: Store new job
  ├─ UPDATE: Change proposal status
  ├─ SELECT: Retrieve historical data
  └─ Complex queries: Analytics/reporting

Tier 3: ClickUp (Business State)

ClickUp serves as the “single source of truth” for pipeline state:

Why ClickUp for state:

Deployment Architecture

Infrastructure

Production Environment:

DigitalOcean Droplet (Ubuntu 22.04 LTS)
├─ 4 vCPU
├─ 8GB RAM
├─ 100GB SSD
└─ Cost: $40/month

Docker Compose Stack:
├─ n8n (primary application)
├─ PostgreSQL 14 (workflow state)
├─ Redis (optional caching layer)
└─ Nginx (reverse proxy + SSL termination)

Service Layout:

Internet
   ↓
Cloudflare (CDN + DDoS protection)
   ↓
Nginx (SSL termination, reverse proxy)
   ↓
┌─────────┬─────────┬─────────┐
│   n8n   │ Postgres│  Redis  │
│  :5678  │  :5432  │  :6379  │
└─────────┴─────────┴─────────┘

High Availability Strategy

Backup & Recovery:

Monitoring:

Disaster Recovery:

Security Measures

Network Security:

Application Security:

Data Security:

Security & Reliability

Error Handling Philosophy

Every workflow follows the pattern:

Try:
  Main workflow logic
Catch (API failure):
  Retry 3x with exponential backoff
Catch (Still failing):
  Send Telegram alert to human
  Log error details
  Continue workflow (don't block other jobs)

Example: Claude API Failure

n8n HTTP Request (Claude API)
   ↓
Error? (429 rate limit)
   ↓
Wait 1s → Retry
   ↓
Still error? (429)
   ↓
Wait 2s → Retry
   ↓
Still error? (429)
   ↓
Wait 4s → Retry
   ↓
Still error?
   ↓
Telegram Alert: "Proposal generation failed for Job XYZ - Review manually"
   ↓
Mark ClickUp task as "Needs Manual Review"
   ↓
Workflow completes (doesn't crash)

Audit Trail

All significant actions are logged:

Storage:

Use Cases:

Performance Optimization

Initial Performance (MVP - Week 1)

Baseline Metrics:

Job Discovery: Email received → Job scored
  ├─ Email fetch (IMAP):        ~2 seconds
  ├─ Job page fetch (HTTP):     ~3 seconds
  ├─ HTML parsing (Cheerio):    ~2 seconds
  ├─ Client history API fetch:  ~5 seconds
  └─ Scoring algorithm:         ~0.5 seconds
  Total: ~12.5 seconds

Proposal Generation: Job qualified → Proposal ready
  ├─ Context building:          ~1 second
  ├─ Claude API call:           ~4 seconds
  ├─ Telegram send:             ~1 second
  └─ ClickUp task creation:     ~2 seconds
  Total: ~8 seconds

Overall Pipeline: Email alert → Proposal ready for approval
  └─ Total: ~20 seconds (excluding 5-min email polling lag)

Bottlenecks Identified:

  1. Client history fetch: 5 seconds (sequential database queries)
  2. Job page parsing: 2 seconds (full Cheerio DOM parse)
  3. ClickUp + Telegram sequential: 3 seconds (could be parallel)

Optimization 1: Parallel API Calls (30% Faster)

Problem: Sequential API calls to ClickUp and Telegram

Before:

// Sequential: Wait for each to complete
await createClickUpTask(data);     // 2 seconds
await sendTelegramMessage(data);   // 1 second
// Total: 3 seconds

After:

// Parallel: Execute simultaneously
await Promise.all([
  createClickUpTask(data),         // 2 seconds
  sendTelegramMessage(data)        // 1 second
]);
// Total: 2 seconds (limited by slowest)

Result: 3 seconds → 2 seconds (1 second saved, 33% faster)

Implementation: n8n “Merge” node with “Wait for All” mode

Optimization 2: Client Data Caching (90% Faster)

Problem: Fetching client history from Upwork on every job (5 seconds)

Analysis:

Solution: Cache client data for 24 hours

Before:

// Every job: Fetch client history
const clientHistory = await upworkAPI.getClientHistory(clientId);
// Time: 5 seconds per job

After:

// Check cache first
let clientHistory = await redis.get(`client:${clientId}`);

if (!clientHistory) {
  // Cache miss: Fetch and store
  clientHistory = await upworkAPI.getClientHistory(clientId);
  await redis.setex(`client:${clientId}`, 86400, clientHistory);
  // Time: 5 seconds (first time only)
} else {
  // Cache hit
  // Time: 0.01 seconds
}

Result:

Implementation: Redis with 24-hour TTL

Optimization 3: Smart Email Parsing (50% Faster)

Problem: Full HTML parsing with Cheerio for every email (2 seconds)

Analysis:

Solution: Regex extraction with Cheerio fallback

Before:

// Full HTML parse every time
const $ = cheerio.load(emailHTML);
const jobUrl = $('a[href*="upwork.com/jobs"]').attr('href');
const title = $('h2.job-title').text();
// Time: 2 seconds

After:

// Try regex first (fast path)
const urlMatch = emailHTML.match(/https:\/\/www\.upwork\.com\/jobs\/~[a-f0-9]+/);
const titleMatch = emailHTML.match(/<h2[^>]*>(.*?)<\/h2>/);

if (urlMatch && titleMatch) {
  // Regex succeeded (95% of cases)
  return { url: urlMatch[0], title: titleMatch[1] };
  // Time: 0.1 seconds
} else {
  // Fallback to Cheerio (complex emails)
  const $ = cheerio.load(emailHTML);
  return { url: $('a').attr('href'), title: $('h2').text() };
  // Time: 2 seconds
}

Result:

Implementation: n8n Function node with conditional logic

Optimization 4: Scoring Algorithm Early Exit (40% Faster)

Problem: Evaluating all 8 scoring factors even when job clearly fails

Solution: Early exit on disqualifying conditions

Before:

function scoreJob(job) {
  let score = 0;
  score += scoreBudget(job.budget);           // Always executes
  score += scoreClient(job.client);            // Always executes
  score += scoreSkills(job.skills);            // Always executes
  score += detectRedFlags(job.description);    // Always executes
  // ... all 8 factors
  return score;
}

After:

function scoreJob(job) {
  // Fast fail: Check disqualifying conditions first
  if (job.budget.amount < MIN_BUDGET) return 0;  // 40% of jobs
  if (hasBlockedKeywords(job.description)) return 0;  // 10% of jobs

  // Only evaluate remaining factors if job passed filters
  let score = 0;
  score += scoreBudget(job.budget);
  score += scoreClient(job.client);
  // ... remaining factors
  return score;
}

Result:

Optimization 5: Claude API Response Caching (Conditional)

Problem: Regenerating proposals for duplicate/similar jobs

Analysis:

Solution: Cache proposals by job content hash (with caution)

Implementation:

// Generate content hash
const jobHash = hashJobContent(job.title, job.description);

// Check cache
let proposal = await redis.get(`proposal:${jobHash}`);

if (!proposal) {
  // Cache miss: Generate new proposal
  proposal = await claudeAPI.generateProposal(job);

  // Cache for 7 days (jobs reposted after this are likely different)
  await redis.setex(`proposal:${jobHash}`, 604800, proposal);
}

// IMPORTANT: Always send for human approval
// (Even cached proposals reviewed before sending)

Result:

Caution: Only cache for clearly identical jobs. When in doubt, regenerate.

Optimization 6: Workflow Execution Parallelization

Problem: n8n workflows executing sequentially when independent

Solution: Split into parallel workflows triggered simultaneously

Before:

Job Qualified
  ↓
Generate Proposal (4s)
  ↓
Create ClickUp Task (2s)
  ↓
Send Telegram (1s)
Total: 7 seconds sequential

After:

Job Qualified
  ├─→ Generate Proposal (4s) ─→ Send Telegram (1s)
  └─→ Create ClickUp Placeholder (2s)
                ↓
         Update when proposal ready (0.5s)
Total: 5 seconds (parallel)

Result: 7s → 5s (29% faster)

Trade-off: Slightly more complex logic (acceptable for speed gain)

Performance Results Summary

Overall Pipeline Performance:

Stage Before (MVP) After (Optimized) Improvement
Job Discovery 12.5s 5s 60% faster
- Email parsing 2s 0.2s 90% faster
- Client data fetch 5s 0.5s 90% faster (cached)
- Scoring 0.5s 0.175s 65% faster (early exit)
Proposal Generation 8s 5s 37.5% faster
- API calls 3s 2s 33% faster (parallel)
Total Pipeline 20.5s 10s 51% faster

Real-World Impact:

Unavoidable Bottlenecks

Cannot Optimize Further:

  1. Email Polling Interval: 5 minutes
    • Why: IMAP protocol limitation, checking more frequently adds load
    • Mitigation: Acceptable lag (not time-critical once in ~10 min range)
    • Alternative: Push email (Gmail API) would be instant but adds complexity
  2. Claude API Latency: 2-4 seconds
    • Why: External service, network + model inference time
    • Can’t change: API response time is what it is
    • Mitigation: Already using fastest model (Sonnet 4.5, not Opus 4.6)
    • Acceptable: Quality worth the wait (better proposals = higher win rate)
  3. Job Page Fetch: 1-2 seconds
    • Why: Upwork server response time (external)
    • Can’t change: Network latency + server processing
    • Mitigation: Already optimized parsing (see Optimization 3)

Total Unavoidable Time: ~7-8 seconds (65% of optimized pipeline)

Conclusion: Further optimization has diminishing returns. Current 10-second pipeline is “good enough” for use case.

Performance Monitoring

Metrics Tracked:

// n8n workflow execution logs
{
  workflow: 'job-discovery',
  duration_ms: 5200,
  stages: {
    email_parse: 180,
    client_fetch: 520,  // Cache hit
    scoring: 175,
    total: 5200
  },
  cache_hits: {
    client_data: true,
    proposal: false
  }
}

Alert Thresholds:

Monthly Performance Review:

Cost-Performance Trade-offs

Redis Caching:

Email Polling Frequency:

Claude Model Selection:

Future Performance Optimizations (Not Implemented)

Considered but not worth ROI:

  1. Job Deduplication:
    • Idea: Detect and skip reposted jobs
    • Savings: ~5% fewer API calls
    • Cost: Complex hashing logic
    • Decision: Not worth complexity for 5% savings
  2. Predictive Pre-fetching:
    • Idea: Pre-fetch likely client data before job arrives
    • Savings: 0.5s average (rare cache misses)
    • Cost: Wasted API calls for jobs never seen
    • Decision: Over-engineering
  3. Edge Computing (Cloudflare Workers):
    • Idea: Run scoring at edge for lower latency
    • Savings: ~100ms (edge is closer)
    • Cost: $5/month + migration effort
    • Decision: 100ms not noticeable to human

Philosophy: Optimize for 80/20 rule. Got 51% faster with simple optimizations. Chasing the remaining 10-15% has diminishing returns.

Performance Considerations (Current State)

Latency Targets (Achieved):

System Load:

Capacity:

Scalability

Current Capacity:

Scaling Paths (if needed):

  1. Horizontal: Add n8n instances behind load balancer
  2. Vertical: Upgrade to larger droplet (more RAM/CPU)
  3. Database: Move PostgreSQL to managed service (RDS)
  4. Caching: Implement Redis for Claude response caching

Cost at Scale:

Future Architecture Enhancements

Potential Improvements:

  1. Vector Database: Store successful proposals, retrieve similar for context
  2. Multi-Platform: Extend to Fiverr, Toptal (same architecture)
  3. Team Mode: Multi-user support (job routing by skills)
  4. Predictive Scoring: ML model to refine scoring algorithm
  5. Auto-Submit: Direct Upwork API integration (if/when available)

Note: This architecture documentation describes the system design and technical decisions. Specific implementation details (workflow configurations, prompt engineering, scoring criteria) are proprietary and not included in this public documentation.

For questions about architectural approach, technology choices, or similar system design challenges, please contact directly.

This site is open source. Improve this page.