XF/GIA
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bbb19f3c2c6431315a3074bb1486c308f552c155
GIA/AGENTS.md

9.7 KiB
Raw Blame History

GIA — Agent Knowledge Base

Overview

GIA is a multi-transport communication platform bridging Signal, WhatsApp, XMPP, and Instagram through a Django web interface. It provides message relay, AI-powered workspace analysis, compose UX, and OSINT search. Stack: Python 3.11, Django 4.x, HTMX, Bulma CSS, SQLite, Redis, Docker Compose. Async runtime uses asyncio + uvloop.

Structure

GIA/
├── app/                  # Django project config (settings, urls, asgi, wsgi)
│   ├── settings.py       # Main settings (imports local_settings.py at bottom)
│   ├── local_settings.py # Env-driven overrides (secrets, feature flags)
│   ├── urls.py           # All URL routing (single flat file, no includes)
│   └── asgi.py           # ASGI entrypoint with WebSocket routing
├── core/                 # ALL application logic lives here
│   ├── models.py         # All models (~1600 lines) — User, Person, AI, Message, etc.
│   ├── forms.py          # Django ModelForms using RestrictedFormMixin from mixins
│   ├── admin.py          # Admin registrations
│   ├── clients/          # Transport service adapters
│   │   ├── __init__.py   # ClientBase ABC (start, message_received, etc.)
│   │   ├── transport.py  # Shared transport layer — attachment prep, send, runtime state
│   │   ├── whatsapp.py   # WhatsApp client via Neonize (~3100 lines)
│   │   ├── signal.py     # Signal client via signal-cli REST API
│   │   ├── xmpp.py       # XMPP client via slixmpp
│   │   ├── instagram.py  # Instagram client via aiograpi
│   │   └── gateway.py    # Gateway HTTP helpers
│   ├── messaging/        # Message processing pipeline
│   │   ├── ai.py         # OpenAI integration (AsyncOpenAI)
│   │   ├── history.py    # Prompt window builder with adaptive limits
│   │   ├── media_bridge.py # Media attachment resolution
│   │   ├── analysis.py   # Conversation analysis
│   │   ├── natural.py    # Natural language processing
│   │   ├── replies.py    # Reply generation
│   │   └── utils.py      # Message formatting helpers
│   ├── modules/
│   │   └── router.py     # UnifiedRouter — orchestrates all transport clients
│   ├── views/            # Django views (class-based)
│   │   ├── compose.py    # Compose UX (~3400 lines) — send, drafts, thread, media
│   │   ├── workspace.py  # AI workspace (~5200 lines) — insights, mitigation, patterns
│   │   ├── osint.py      # OSINT/search interface
│   │   └── ...           # CRUD views for people, groups, sessions, etc.
│   ├── lib/prompts/      # AI persona prompt templates
│   ├── realtime/         # WebSocket handlers (compose thread)
│   ├── templates/        # Django templates (75 files, partials/ heavy)
│   ├── management/commands/ # ur (unified router), scheduling
│   └── util/logs.py      # Custom colored logger — use logs.get_logger("name")
├── Makefile              # Docker Compose orchestration commands
├── docker-compose.yml    # Services: app, asgi, ur, scheduling, redis, signal-cli
├── Dockerfile            # Python 3.11, venv at /venv
├── requirements.txt      # Pinned deps (django, openai, neonize, slixmpp, etc.)
├── stack.env             # Runtime env vars (from stack.env.example)
└── LLM_CODING_STANDARDS.md # Project-specific coding rules (READ THIS)

Commands

# All commands run via Docker Compose with stack.env
make build                          # Build Docker images
make run                            # Start all services (quadlet manager)
make stop                           # Stop all services
make log                            # Tail logs
make compose-run                    # Start via docker-compose directly
make compose-stop                   # Stop via docker-compose
make compose-log                    # Tail via docker-compose

# Database
make migrate                        # Run Django migrations
make makemigrations                 # Generate new migrations
make auth                           # Create superuser

# Testing
make test                           # Run all tests
make test MODULES=core.tests        # Run specific test module
# Inside container (or with venv activated):
python manage.py test core.tests -v 2              # All tests
python manage.py test core.tests.test_foo -v 2     # Single test module
python manage.py test core.tests.test_foo.TestBar -v 2  # Single class
python manage.py test core.tests.test_foo.TestBar.test_method -v 2  # Single test

# Service restarts after code changes
docker-compose restart ur           # Restart unified router
docker-compose restart scheduling   # Restart scheduler
# uWSGI auto-reloads for app/core code changes

Code Style

Formatting & Linting (pre-commit enforced)

  • Black: Line length 88, excludes core/migrations/
  • isort: Profile black (compatible grouping)
  • flake8: Max line 88, ignores E203, E231, E501, E702, W291
  • djhtml: Template indent 2 spaces (-t 2)
  • ripsecrets: Scans for leaked credentials

Imports

  • Standard library first, then third-party, then Django, then project
  • Project imports use absolute paths: from core.models import Person
  • Within same package, relative OK: from .models import User (seen in admin/forms)
  • Views import models explicitly: from core.models import AI, Person, Message

Naming

  • snake_case for functions, variables, modules
  • PascalCase for classes (Django views, models)
  • UPPER_CASE for module-level constants and Django settings
  • Private helpers prefixed with _: _safe_limit(), _service_key()
  • Service names always lowercase strings: "signal", "whatsapp", "xmpp", "instagram"

Logging

  • Use from core.util import logs then log = logs.get_logger("name") for transport/messaging code
  • Use import logging; logger = logging.getLogger(__name__) for views/models
  • Info for lifecycle events, warning/error for failures, debug for high-volume traces
  • Debug logs gated behind GIA_DEBUG_LOGS env var

Views

  • Class-based views inheriting View or LoginRequiredMixin
  • CRUD views use mixins.views.ObjectCreate/ObjectUpdate/ObjectDelete from django-crud-mixins
  • Forms use RestrictedFormMixin for user-scoped queryset filtering
  • HTMX-driven partials in core/templates/partials/

Models

  • All models in single core/models.py (~1600 lines)
  • UUIDs as primary keys for Person and related models
  • SERVICE_CHOICES tuple for transport type fields
  • Custom User model extending AbstractUser with billing fields
  • Multi-tenant: most models have user = ForeignKey(User)

Async Patterns

  • Transport clients are async (async def send_message_raw(...))
  • Views bridge sync Django to async transport via async_to_sync()
  • orjson for fast JSON serialization in transport layer
  • Redis cache for runtime state, bridge maps, command queues

Error Handling

  • Standard try/except with specific exception types
  • Django ValidationError for model validation
  • get_object_or_404() in views for missing resources
  • HttpResponseBadRequest / HttpResponseNotFound for view error responses
  • No custom exception hierarchy — use built-in Django/Python exceptions

LLM Coding Standards (from LLM_CODING_STANDARDS.md)

MUST READ: LLM_CODING_STANDARDS.md contains binding project rules. Key points:

  • Fix root causes; don't paper over with UI-only patches
  • Keep behavior symmetric across all transports where protocol permits
  • Centralize shared logic — no copy/paste service forks
  • Shared attachment prep goes through core/clients/transport.py
  • Never inject internal blob links as relay body text for attachment-only messages
  • After changing core/clients/* or router/relay/transport: restart runtime (make stop && make run)
  • Logging: lifecycle at info, failures at warning/error, high-volume at debug
  • Debug diagnostics must be gated (e.g. WHATSAPP_DEBUG) and removable in one patch
  • When touching large files (2000+ lines): extract minimal reusable helpers, add docstrings
  • Update INSTALL.md and README.md when operational commands/env requirements change

Anti-Patterns

  • DO NOT create separate transport-specific media pipelines — use transport.prepare_outbound_attachments()
  • DO NOT add TODO/FIXME comments — codebase is currently clean of them
  • DO NOT use print() — use the logging system via logs.get_logger()
  • DO NOT modify core/migrations/ files — Black/linting excludes them for a reason
  • DO NOT commit stack.env, db.sqlite3, or any secrets — ripsecrets pre-commit hook will block
  • DO NOT add new models outside core/models.py — all models live in one file
  • DO NOT use type suppression or ignore runtime errors silently

Key Architecture Notes

  • Unified Router (core/modules/router.py): Management command python manage.py ur runs the event loop with all transport clients. Each client inherits ClientBase ABC.
  • Transport Layer (core/clients/transport.py): Shared cache-backed runtime state, command queuing, and attachment prep. All outbound media goes through prepare_outbound_attachments().
  • Settings Chain: app/settings.py → imports app/local_settings.py (wildcard *) → env vars from stack.env. Feature flags: WHATSAPP_ENABLED, INSTAGRAM_ENABLED, COMPOSE_WS_ENABLED.
  • Services in docker-compose: app (uWSGI), asgi (uvicorn for WebSockets), ur (unified router), scheduling (APScheduler), redis, signal-cli-rest-api.
  • No test suite currently: core/tests.py is empty scaffold; core/tests/ has only __init__.py. Tests run via make test MODULES=... but need to be written.