XF/GIA
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Harden security

Browse Source
This commit is contained in:
Mark Veidemanis
2026-03-05 05:42:19 +00:00
parent 06735bdfb1
commit 438e561da0
75 changed files with 6260 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
AGENTS.md
View File

@@ -2,7 +2,7 @@
## Overview ## Overview
GIA is a multi-transport communication platform bridging Signal, WhatsApp, XMPP, and Instagram through a Django web interface. It provides message relay, AI-powered workspace analysis, compose UX, and OSINT search. Stack: Python 3.11, Django 4.x, HTMX, Bulma CSS, SQLite, Redis, Docker Compose. Async runtime uses asyncio + uvloop. GIA is a multi-transport communication platform bridging Signal, WhatsApp, XMPP, and Instagram through a Django web interface. It provides message relay, AI-powered workspace analysis, compose UX, and OSINT search. Stack: Python 3.11, Django 4.x, HTMX, Bulma CSS, SQLite, Redis, Podman. Async runtime uses asyncio + uvloop.
## Structure ## Structure
@@ -45,8 +45,8 @@ GIA/
│ ├── templates/ # Django templates (75 files, partials/ heavy) │ ├── templates/ # Django templates (75 files, partials/ heavy)
│ ├── management/commands/ # ur (unified router), scheduling │ ├── management/commands/ # ur (unified router), scheduling
│ └── util/logs.py # Custom colored logger — use logs.get_logger("name") │ └── util/logs.py # Custom colored logger — use logs.get_logger("name")
├── Makefile # Docker Compose orchestration commands ├── Makefile # Podman + quadlet orchestration commands
├── docker-compose.yml # Services: app, asgi, ur, scheduling, redis, signal-cli ├── scripts/quadlet/ # Podman lifecycle scripts and unit rendering
├── Dockerfile # Python 3.11, venv at /venv ├── Dockerfile # Python 3.11, venv at /venv
├── requirements.txt # Pinned deps (django, openai, neonize, slixmpp, etc.) ├── requirements.txt # Pinned deps (django, openai, neonize, slixmpp, etc.)
├── stack.env # Runtime env vars (from stack.env.example) ├── stack.env # Runtime env vars (from stack.env.example)
@@ -56,14 +56,11 @@ GIA/
## Commands ## Commands
```bash ```bash
# All commands run via Docker Compose with stack.env # All commands run via Podman + quadlet with stack.env
make build # Build Docker images make build # Build Docker images
make run # Start all services (quadlet manager) make run # Start all services (quadlet manager)
make stop # Stop all services make stop # Stop all services
make log # Tail logs make log # Tail logs
make compose-run # Start via docker-compose directly
make compose-stop # Stop via docker-compose
make compose-log # Tail via docker-compose
# Database # Database
make migrate # Run Django migrations make migrate # Run Django migrations
@@ -80,8 +77,8 @@ python manage.py test core.tests.test_foo.TestBar -v 2 # Single class
python manage.py test core.tests.test_foo.TestBar.test_method -v 2 # Single test python manage.py test core.tests.test_foo.TestBar.test_method -v 2 # Single test
# Service restarts after code changes # Service restarts after code changes
docker-compose restart ur # Restart unified router podman restart ur_gia # Restart unified router
docker-compose restart scheduling # Restart scheduler podman restart scheduling_gia # Restart scheduler
# uWSGI auto-reloads for app/core code changes # uWSGI auto-reloads for app/core code changes
``` ```
@@ -169,5 +166,5 @@ docker-compose restart scheduling # Restart scheduler
- **Unified Router** (`core/modules/router.py`): Management command `python manage.py ur` runs the event loop with all transport clients. Each client inherits `ClientBase` ABC. - **Unified Router** (`core/modules/router.py`): Management command `python manage.py ur` runs the event loop with all transport clients. Each client inherits `ClientBase` ABC.
- **Transport Layer** (`core/clients/transport.py`): Shared cache-backed runtime state, command queuing, and attachment prep. All outbound media goes through `prepare_outbound_attachments()`. - **Transport Layer** (`core/clients/transport.py`): Shared cache-backed runtime state, command queuing, and attachment prep. All outbound media goes through `prepare_outbound_attachments()`.
- **Settings Chain**: `app/settings.py` → imports `app/local_settings.py` (wildcard `*`) → env vars from `stack.env`. Feature flags: `WHATSAPP_ENABLED`, `INSTAGRAM_ENABLED`, `COMPOSE_WS_ENABLED`. - **Settings Chain**: `app/settings.py` → imports `app/local_settings.py` (wildcard `*`) → env vars from `stack.env`. Feature flags: `WHATSAPP_ENABLED`, `INSTAGRAM_ENABLED`, `COMPOSE_WS_ENABLED`.
- **Services in docker-compose**: `app` (uWSGI), `asgi` (uvicorn for WebSockets), `ur` (unified router), `scheduling` (APScheduler), `redis`, `signal-cli-rest-api`. - **Services in podman stack**: `app` (uWSGI), `asgi` (uvicorn for WebSockets), `ur` (unified router), `scheduling` (APScheduler), `redis`, `signal-cli-rest-api`.
- **No test suite currently**: `core/tests.py` is empty scaffold; `core/tests/` has only `__init__.py`. Tests run via `make test MODULES=...` but need to be written. - **No test suite currently**: `core/tests.py` is empty scaffold; `core/tests/` has only `__init__.py`. Tests run via `make test MODULES=...` but need to be written.

8
Dockerfile
View File

@@ -14,17 +14,13 @@ RUN chown xf:xf /venv
RUN apt-get update && apt-get install -y cargo rustc RUN apt-get update && apt-get install -y cargo rustc
USER xf
ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1 ENV PYTHONUNBUFFERED=1
WORKDIR /code WORKDIR /code
COPY requirements.txt /code/ COPY requirements.txt /code/
RUN python -m venv /venv RUN python -m venv /venv
RUN . /venv/bin/activate && pip install -r requirements.txt RUN . /venv/bin/activate && pip install -r requirements.txt
RUN chown -R xf:xf /code /venv /conf
# CMD . /venv/bin/activate && uwsgi --ini /conf/uwsgi.ini USER xf
CMD if [ "$OPERATION" = "uwsgi" ] ; then . /venv/bin/activate && uwsgi --ini /conf/uwsgi.ini ; else . /venv/bin/activate && exec python manage.py runserver 0.0.0.0:8000; fi CMD if [ "$OPERATION" = "uwsgi" ] ; then . /venv/bin/activate && uwsgi --ini /conf/uwsgi.ini ; else . /venv/bin/activate && exec python manage.py runserver 0.0.0.0:8000; fi
# CMD . /venv/bin/activate && uvicorn --reload --reload-include *.html --workers 2 --uds /var/run/socks/app.sock app.asgi:application
# CMD . /venv/bin/activate && gunicorn -b 0.0.0.0:8000 --reload app.asgi:application -k uvicorn.workers.UvicornWorker

8
INSTALL.md
View File

@@ -6,7 +6,7 @@ Use this first. Then read `README.md` for feature and operation-mode details.
## 1) Prerequisites ## 1) Prerequisites
- Linux host with either Podman + podman-compose wrapper or Docker Compose compatibility. - Linux host with Podman.
- Git. - Git.
- Network access for service images and Python dependencies. - Network access for service images and Python dependencies.
@@ -99,7 +99,7 @@ make log
Basic stack status: Basic stack status:
```bash ```bash
docker-compose --env-file=stack.env ps make status
``` ```
## 7) Restart conventions ## 7) Restart conventions
@@ -115,7 +115,7 @@ Use the explicit `make stop && make run` command sequence when a full recycle is
### Single service restart ### Single service restart
```bash ```bash
docker-compose --env-file=stack.env restart <service> podman restart <container-name>
``` ```
If single-service restart fails due to dependency/pod state, use full recycle above. If single-service restart fails due to dependency/pod state, use full recycle above.
@@ -127,7 +127,7 @@ After changing UR/runtime code (`core/clients/*`, transport, relay paths), resta
Minimum target: Minimum target:
```bash ```bash
docker-compose --env-file=stack.env restart ur podman restart ur_gia
``` ```
If blocked, use full recycle. If blocked, use full recycle.

78
Makefile
View File

@@ -1,15 +1,14 @@
QUADLET_MGR := ./scripts/quadlet/manage.sh QUADLET_MGR := ./scripts/quadlet/manage.sh
MODULES ?= core.tests MODULES ?= core.tests
STACK_ID_CLEAN := $(shell sid="$${GIA_STACK_ID:-$${STACK_ID:-}}"; sid=$$(printf "%s" "$$sid" | tr -cs 'a-zA-Z0-9._-' '-' | sed 's/^-*//; s/-*$$//'); printf "%s" "$$sid")
STACK_SUFFIX := $(if $(STACK_ID_CLEAN),_$(STACK_ID_CLEAN),)
APP_CONTAINER := gia$(STACK_SUFFIX)
run: run:
bash $(QUADLET_MGR) up bash $(QUADLET_MGR) up
build: build:
@if command -v docker-compose >/dev/null 2>&1; then \ OPERATION=uwsgi podman build --build-arg OPERATION=uwsgi -t localhost/xf/gia:prod -f Dockerfile .
docker-compose --env-file=stack.env build app; \
else \
OPERATION=uwsgi podman build --build-arg OPERATION=uwsgi -t localhost/xf/gia:prod -f Dockerfile .; \
fi
stop: stop:
bash $(QUADLET_MGR) down bash $(QUADLET_MGR) down
@@ -23,71 +22,42 @@ status:
quadlet-install: quadlet-install:
bash $(QUADLET_MGR) install bash $(QUADLET_MGR) install
compose-run:
docker-compose --env-file=stack.env up -d
compose-stop:
docker-compose --env-file=stack.env down
compose-log:
docker-compose --env-file=stack.env logs -f --names
test: test:
@if command -v docker-compose >/dev/null 2>&1; then \ @if podman ps --format '{{.Names}}' | grep -qx "$(APP_CONTAINER)"; then \
docker-compose --env-file=stack.env run --rm app sh -c ". /venv/bin/activate && python manage.py test $(MODULES) -v 2"; \ podman exec "$(APP_CONTAINER)" sh -lc "cd /code && . /venv/bin/activate && python manage.py test $(MODULES) -v 2"; \
else \ else \
if podman ps --format '{{.Names}}' | grep -qx gia; then \ echo "Container '$(APP_CONTAINER)' is not running. Start the stack first with 'make run'." >&2; \
podman exec gia sh -lc "cd /code && . /venv/bin/activate && python manage.py test $(MODULES) -v 2"; \ exit 125; \
else \
echo "Container 'gia' is not running. Start the stack first with 'make run' (or mrl)." >&2; \
exit 125; \
fi; \
fi fi
migrate: migrate:
@if command -v docker-compose >/dev/null 2>&1; then \ @if podman ps --format '{{.Names}}' | grep -qx "$(APP_CONTAINER)"; then \
docker-compose --env-file=stack.env run --rm app sh -c ". /venv/bin/activate && python manage.py migrate"; \ podman exec "$(APP_CONTAINER)" sh -lc "cd /code && . /venv/bin/activate && python manage.py migrate"; \
else \ else \
if podman ps --format '{{.Names}}' | grep -qx gia; then \ echo "Container '$(APP_CONTAINER)' is not running. Start the stack first with 'make run'." >&2; \
podman exec gia sh -lc "cd /code && . /venv/bin/activate && python manage.py migrate"; \ exit 125; \
else \
echo "Container 'gia' is not running. Start the stack first with 'make run' (or mrl)." >&2; \
exit 125; \
fi; \
fi fi
makemigrations: makemigrations:
@if command -v docker-compose >/dev/null 2>&1; then \ @if podman ps --format '{{.Names}}' | grep -qx "$(APP_CONTAINER)"; then \
docker-compose --env-file=stack.env run --rm app sh -c ". /venv/bin/activate && python manage.py makemigrations"; \ podman exec "$(APP_CONTAINER)" sh -lc "cd /code && . /venv/bin/activate && python manage.py makemigrations"; \
else \ else \
if podman ps --format '{{.Names}}' | grep -qx gia; then \ echo "Container '$(APP_CONTAINER)' is not running. Start the stack first with 'make run'." >&2; \
podman exec gia sh -lc "cd /code && . /venv/bin/activate && python manage.py makemigrations"; \ exit 125; \
else \
echo "Container 'gia' is not running. Start the stack first with 'make run' (or mrl)." >&2; \
exit 125; \
fi; \
fi fi
auth: auth:
@if command -v docker-compose >/dev/null 2>&1; then \ @if podman ps --format '{{.Names}}' | grep -qx "$(APP_CONTAINER)"; then \
docker-compose --env-file=stack.env run --rm app sh -c ". /venv/bin/activate && python manage.py createsuperuser"; \ podman exec "$(APP_CONTAINER)" sh -lc "cd /code && . /venv/bin/activate && python manage.py createsuperuser"; \
else \ else \
if podman ps --format '{{.Names}}' | grep -qx gia; then \ echo "Container '$(APP_CONTAINER)' is not running. Start the stack first with 'make run'." >&2; \
podman exec gia sh -lc "cd /code && . /venv/bin/activate && python manage.py createsuperuser"; \ exit 125; \
else \
echo "Container 'gia' is not running. Start the stack first with 'make run' (or mrl)." >&2; \
exit 125; \
fi; \
fi fi
token: token:
@if command -v docker-compose >/dev/null 2>&1; then \ @if podman ps --format '{{.Names}}' | grep -qx "$(APP_CONTAINER)"; then \
docker-compose --env-file=stack.env run --rm app sh -c ". /venv/bin/activate && python manage.py addstatictoken m"; \ podman exec "$(APP_CONTAINER)" sh -lc "cd /code && . /venv/bin/activate && python manage.py addstatictoken m"; \
else \ else \
if podman ps --format '{{.Names}}' | grep -qx gia; then \ echo "Container '$(APP_CONTAINER)' is not running. Start the stack first with 'make run'." >&2; \
podman exec gia sh -lc "cd /code && . /venv/bin/activate && python manage.py addstatictoken m"; \ exit 125; \
else \
echo "Container 'gia' is not running. Start the stack first with 'make run' (or mrl)." >&2; \
exit 125; \
fi; \
fi fi

25
app/local_settings.py
View File

@@ -7,7 +7,7 @@ DOMAIN = getenv("DOMAIN", "example.com")
URL = getenv("URL", f"https://{DOMAIN}") URL = getenv("URL", f"https://{DOMAIN}")
# Access control # Access control
ALLOWED_HOSTS = getenv("ALLOWED_HOSTS", f"127.0.0.1,{DOMAIN}").split(",") ALLOWED_HOSTS = getenv("ALLOWED_HOSTS", f"localhost,{DOMAIN}").split(",")
# CSRF # CSRF
CSRF_TRUSTED_ORIGINS = getenv("CSRF_TRUSTED_ORIGINS", URL).split(",") CSRF_TRUSTED_ORIGINS = getenv("CSRF_TRUSTED_ORIGINS", URL).split(",")
@@ -40,10 +40,14 @@ if DEBUG:
import socket # only if you haven't already imported this import socket # only if you haven't already imported this
hostname, _, ips = socket.gethostbyname_ex(socket.gethostname()) hostname, _, ips = socket.gethostbyname_ex(socket.gethostname())
INTERNAL_IPS = [ip[: ip.rfind(".")] + ".1" for ip in ips] + [ INTERNAL_IPS = [ip[: ip.rfind(".")] + ".1" for ip in ips]
"127.0.0.1", INTERNAL_IPS.extend(
"10.0.2.2", [
] item.strip()
for item in getenv("DEBUG_INTERNAL_IPS", "localhost").split(",")
if item.strip()
]
)
SETTINGS_EXPORT = ["BILLING_ENABLED"] SETTINGS_EXPORT = ["BILLING_ENABLED"]
@@ -69,6 +73,15 @@ TRACE_PROPAGATION_ENABLED = getenv("TRACE_PROPAGATION_ENABLED", "true").lower()
EVENT_PRIMARY_WRITE_PATH = getenv("EVENT_PRIMARY_WRITE_PATH", "false").lower() in trues EVENT_PRIMARY_WRITE_PATH = getenv("EVENT_PRIMARY_WRITE_PATH", "false").lower() in trues
MEMORY_SEARCH_BACKEND = getenv("MEMORY_SEARCH_BACKEND", "django") MEMORY_SEARCH_BACKEND = getenv("MEMORY_SEARCH_BACKEND", "django")
MANTICORE_HTTP_URL = getenv("MANTICORE_HTTP_URL", "http://127.0.0.1:9308") MANTICORE_HTTP_URL = getenv("MANTICORE_HTTP_URL", "http://localhost:9308")
MANTICORE_MEMORY_TABLE = getenv("MANTICORE_MEMORY_TABLE", "gia_memory_items") MANTICORE_MEMORY_TABLE = getenv("MANTICORE_MEMORY_TABLE", "gia_memory_items")
MANTICORE_HTTP_TIMEOUT = int(getenv("MANTICORE_HTTP_TIMEOUT", "5") or 5) MANTICORE_HTTP_TIMEOUT = int(getenv("MANTICORE_HTTP_TIMEOUT", "5") or 5)
# Attachment security defaults for transport adapters.
ATTACHMENT_MAX_BYTES = int(getenv("ATTACHMENT_MAX_BYTES", str(25 * 1024 * 1024)) or 0)
ATTACHMENT_ALLOW_PRIVATE_URLS = (
getenv("ATTACHMENT_ALLOW_PRIVATE_URLS", "false").lower() in trues
)
ATTACHMENT_ALLOW_UNKNOWN_MIME = (
getenv("ATTACHMENT_ALLOW_UNKNOWN_MIME", "false").lower() in trues
)

5
app/settings.py
View File

@@ -189,8 +189,9 @@ REST_FRAMEWORK = {
} }
INTERNAL_IPS = [ INTERNAL_IPS = [
"127.0.0.1", item.strip()
"10.1.10.11", for item in os.getenv("INTERNAL_IPS", "localhost").split(",")
if item.strip()
] ]
DEBUG_TOOLBAR_PANELS = [ DEBUG_TOOLBAR_PANELS = [

1488
artifacts/audits/1-initial.json Normal file
View File

File diff suppressed because it is too large Load Diff

674
artifacts/audits/2-first-pass-fix.json Normal file
View File

@@ -0,0 +1,674 @@
{
"score": 58,
"grade": "D",
"gradeLabel": "Significant security risks",
"totalFindings": 22,
"totalDepVulns": 0,
"categories": {
"secrets": {
"label": "Secrets",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"injection": {
"label": "Code Vulnerabilities",
"findingCount": 3,
"deduction": 9,
"counts": {
"critical": 0,
"high": 0,
"medium": 3,
"low": 0
}
},
"deps": {
"label": "Dependencies",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"auth": {
"label": "Auth & Access Control",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"config": {
"label": "Configuration",
"findingCount": 4,
"deduction": 10,
"counts": {
"critical": 0,
"high": 4,
"medium": 0,
"low": 0
}
},
"supply-chain": {
"label": "Supply Chain",
"findingCount": 1,
"deduction": 3,
"counts": {
"critical": 0,
"high": 0,
"medium": 1,
"low": 0
}
},
"api": {
"label": "API Security",
"findingCount": 3,
"deduction": 10,
"counts": {
"critical": 0,
"high": 3,
"medium": 0,
"low": 0
}
},
"llm": {
"label": "AI/LLM Security",
"findingCount": 11,
"deduction": 10,
"counts": {
"critical": 0,
"high": 0,
"medium": 11,
"low": 0
}
}
},
"findings": [
{
"file": "/code/xf/GIA/Dockerfile",
"line": 25,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/Dockerfile",
"line": 28,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/Dockerfile",
"line": 30,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/Dockerfile",
"line": 31,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/core/clients/whatsapp.py",
"line": 3398,
"severity": "high",
"category": "api",
"rule": "API_UPLOAD_NO_TYPE_CHECK",
"title": "API: File Upload Without Type Validation",
"description": "File upload using original filename without type validation.",
"fix": "Validate file extension and MIME type. Generate random filenames for storage.",
"cwe": "CWE-434",
"owasp": "A04:2021"
},
{
"file": "/code/xf/GIA/core/clients/xmpp.py",
"line": 57,
"severity": "high",
"category": "api",
"rule": "API_UPLOAD_NO_TYPE_CHECK",
"title": "API: File Upload Without Type Validation",
"description": "File upload using original filename without type validation.",
"fix": "Validate file extension and MIME type. Generate random filenames for storage.",
"cwe": "CWE-434",
"owasp": "A04:2021"
},
{
"file": "/code/xf/GIA/core/security/attachments.py",
"line": 83,
"severity": "high",
"category": "api",
"rule": "API_UPLOAD_NO_TYPE_CHECK",
"title": "API: File Upload Without Type Validation",
"description": "File upload using original filename without type validation.",
"fix": "Validate file extension and MIME type. Generate random filenames for storage.",
"cwe": "CWE-434",
"owasp": "A04:2021"
},
{
"file": "/code/xf/GIA/core/tests/test_attachment_security.py",
"line": 29,
"severity": "medium",
"category": "ssrf",
"rule": "SSRF_INTERNAL_IP",
"title": "SSRF: Internal IP Pattern",
"description": "Internal IP address in code. Verify it is not reachable via user-controlled URLs.",
"fix": "Block private IP ranges in URL validation for user-supplied URLs",
"cwe": "CWE-918",
"owasp": "A10:2021"
},
{
"file": "/code/xf/GIA/core/tests/test_attachment_security.py",
"line": 34,
"severity": "medium",
"category": "ssrf",
"rule": "SSRF_INTERNAL_IP",
"title": "SSRF: Internal IP Pattern",
"description": "Internal IP address in code. Verify it is not reachable via user-controlled URLs.",
"fix": "Block private IP ranges in URL validation for user-supplied URLs",
"cwe": "CWE-918",
"owasp": "A10:2021"
},
{
"file": "/code/xf/GIA/core/tests/test_attachment_security.py",
"line": 35,
"severity": "medium",
"category": "ssrf",
"rule": "SSRF_INTERNAL_IP",
"title": "SSRF: Internal IP Pattern",
"description": "Internal IP address in code. Verify it is not reachable via user-controlled URLs.",
"fix": "Block private IP ranges in URL validation for user-supplied URLs",
"cwe": "CWE-918",
"owasp": "A10:2021"
},
{
"file": "/code/xf/GIA/requirements.txt",
"line": 23,
"severity": "medium",
"category": "supply-chain",
"rule": "UNPINNED_PYTHON_DEP",
"title": "Unpinned Python Dependency: ./vendor/django-crud-mixins",
"description": "Python dependency without version pin. Pin to a specific version for reproducible builds.",
"fix": "Pin version: ./vendor/django-crud-mixins==x.y.z",
"cwe": null,
"owasp": null
},
{
"file": "/code/xf/GIA/core/clients/signalapi.py",
"line": 411,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 739,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 744,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 758,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 850,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1377,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1382,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1396,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/signal.py",
"line": 189,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
},
{
"file": "/code/xf/GIA/core/views/signal.py",
"line": 197,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
},
{
"file": "/code/xf/GIA/core/views/signal.py",
"line": 206,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
}
],
"depVulns": [],
"remediationPlan": [
{
"priority": 1,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:25",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 2,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:28",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 3,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:30",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 4,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:31",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 5,
"severity": "high",
"category": "api",
"categoryLabel": "API SECURITY",
"title": "API: File Upload Without Type Validation",
"file": "core/clients/whatsapp.py:3398",
"action": "Validate file extension and MIME type. Generate random filenames for storage.",
"effort": "medium"
},
{
"priority": 6,
"severity": "high",
"category": "api",
"categoryLabel": "API SECURITY",
"title": "API: File Upload Without Type Validation",
"file": "core/clients/xmpp.py:57",
"action": "Validate file extension and MIME type. Generate random filenames for storage.",
"effort": "medium"
},
{
"priority": 7,
"severity": "high",
"category": "api",
"categoryLabel": "API SECURITY",
"title": "API: File Upload Without Type Validation",
"file": "core/security/attachments.py:83",
"action": "Validate file extension and MIME type. Generate random filenames for storage.",
"effort": "medium"
},
{
"priority": 8,
"severity": "medium",
"category": "ssrf",
"categoryLabel": "SSRF",
"title": "SSRF: Internal IP Pattern",
"file": "core/tests/test_attachment_security.py:29",
"action": "Block private IP ranges in URL validation for user-supplied URLs",
"effort": "medium"
},
{
"priority": 9,
"severity": "medium",
"category": "ssrf",
"categoryLabel": "SSRF",
"title": "SSRF: Internal IP Pattern",
"file": "core/tests/test_attachment_security.py:34",
"action": "Block private IP ranges in URL validation for user-supplied URLs",
"effort": "medium"
},
{
"priority": 10,
"severity": "medium",
"category": "ssrf",
"categoryLabel": "SSRF",
"title": "SSRF: Internal IP Pattern",
"file": "core/tests/test_attachment_security.py:35",
"action": "Block private IP ranges in URL validation for user-supplied URLs",
"effort": "medium"
},
{
"priority": 11,
"severity": "medium",
"category": "supply-chain",
"categoryLabel": "SUPPLY CHAIN",
"title": "Unpinned Python Dependency: ./vendor/django-crud-mixins",
"file": "requirements.txt:23",
"action": "Pin version: ./vendor/django-crud-mixins==x.y.z",
"effort": "medium"
},
{
"priority": 12,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/clients/signalapi.py:411",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
},
{
"priority": 13,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:739",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 14,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:744",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 15,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:758",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 16,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:850",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 17,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1377",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 18,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1382",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 19,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1396",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 20,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/views/signal.py:189",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
},
{
"priority": 21,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/views/signal.py:197",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
},
{
"priority": 22,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/views/signal.py:206",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
}
],
"recon": {
"frameworks": [
"django"
],
"languages": [
"python"
],
"apiRoutes": [
"app/urls.py",
"core/management/commands/backfill_xmpp_attachment_urls.py"
],
"authPatterns": [],
"databases": [],
"cloudProviders": [],
"frontendExposure": [],
"packageManagers": [
"pip"
],
"cicd": [],
"hasDockerfile": true,
"hasTerraform": false,
"hasKubernetes": false,
"envFiles": [],
"configFiles": []
},
"agents": [
{
"agent": "InjectionTester",
"category": "injection",
"findingCount": 0,
"success": true
},
{
"agent": "AuthBypassAgent",
"category": "auth",
"findingCount": 0,
"success": true
},
{
"agent": "SSRFProber",
"category": "ssrf",
"findingCount": 3,
"success": true
},
{
"agent": "SupplyChainAudit",
"category": "supply-chain",
"findingCount": 1,
"success": true
},
{
"agent": "ConfigAuditor",
"category": "config",
"findingCount": 4,
"success": true
},
{
"agent": "LLMRedTeam",
"category": "llm",
"findingCount": 11,
"success": true
},
{
"agent": "MobileScanner",
"category": "mobile",
"findingCount": 0,
"success": true
},
{
"agent": "GitHistoryScanner",
"category": "history",
"findingCount": 0,
"success": true
},
{
"agent": "CICDScanner",
"category": "cicd",
"findingCount": 0,
"success": true
},
{
"agent": "APIFuzzer",
"category": "api",
"findingCount": 3,
"success": true
}
]
}

410
artifacts/audits/3-second-pass-fix.json Normal file
View File

@@ -0,0 +1,410 @@
{
"score": 74,
"grade": "C",
"gradeLabel": "Fix before shipping",
"totalFindings": 10,
"totalDepVulns": 0,
"categories": {
"secrets": {
"label": "Secrets",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"injection": {
"label": "Code Vulnerabilities",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"deps": {
"label": "Dependencies",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"auth": {
"label": "Auth & Access Control",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"config": {
"label": "Configuration",
"findingCount": 1,
"deduction": 8,
"counts": {
"critical": 0,
"high": 1,
"medium": 0,
"low": 0
}
},
"supply-chain": {
"label": "Supply Chain",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"api": {
"label": "API Security",
"findingCount": 1,
"deduction": 8,
"counts": {
"critical": 0,
"high": 1,
"medium": 0,
"low": 0
}
},
"llm": {
"label": "AI/LLM Security",
"findingCount": 8,
"deduction": 10,
"counts": {
"critical": 0,
"high": 0,
"medium": 8,
"low": 0
}
}
},
"findings": [
{
"file": "/code/xf/GIA/Dockerfile",
"line": 26,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/core/security/attachments.py",
"line": 113,
"severity": "high",
"category": "api",
"rule": "API_UPLOAD_NO_TYPE_CHECK",
"title": "API: File Upload Without Type Validation",
"description": "File upload using original filename without type validation.",
"fix": "Validate file extension and MIME type. Generate random filenames for storage.",
"cwe": "CWE-434",
"owasp": "A04:2021"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 775,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 781,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 795,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1418,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1424,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/osint.py",
"line": 1438,
"severity": "medium",
"category": "llm",
"rule": "LLM_RAG_NO_VALIDATION",
"title": "RAG Pipeline Without Input Validation",
"description": "User input passed directly to vector search/embedding without validation.",
"fix": "Validate and sanitize input before embedding. Limit query length.",
"cwe": "CWE-20",
"owasp": "LLM08"
},
{
"file": "/code/xf/GIA/core/views/signal.py",
"line": 202,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
},
{
"file": "/code/xf/GIA/core/views/signal.py",
"line": 211,
"severity": "medium",
"category": "llm",
"rule": "LLM_NO_OUTPUT_FILTER",
"title": "LLM Output Without Filtering",
"description": "LLM output used directly without filtering. May contain sensitive info or hallucinations.",
"fix": "Filter LLM output before displaying: remove PII, validate against expected format",
"cwe": "CWE-200",
"owasp": "LLM02"
}
],
"depVulns": [],
"remediationPlan": [
{
"priority": 1,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:26",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 2,
"severity": "high",
"category": "api",
"categoryLabel": "API SECURITY",
"title": "API: File Upload Without Type Validation",
"file": "core/security/attachments.py:113",
"action": "Validate file extension and MIME type. Generate random filenames for storage.",
"effort": "medium"
},
{
"priority": 3,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:775",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 4,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:781",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 5,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:795",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 6,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1418",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 7,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1424",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 8,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "RAG Pipeline Without Input Validation",
"file": "core/views/osint.py:1438",
"action": "Validate and sanitize input before embedding. Limit query length.",
"effort": "high"
},
{
"priority": 9,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/views/signal.py:202",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
},
{
"priority": 10,
"severity": "medium",
"category": "llm",
"categoryLabel": "AI/LLM SECURITY",
"title": "LLM Output Without Filtering",
"file": "core/views/signal.py:211",
"action": "Filter LLM output before displaying: remove PII, validate against expected format",
"effort": "high"
}
],
"recon": {
"frameworks": [
"django"
],
"languages": [
"python"
],
"apiRoutes": [
"app/urls.py",
"core/management/commands/backfill_xmpp_attachment_urls.py"
],
"authPatterns": [],
"databases": [],
"cloudProviders": [],
"frontendExposure": [],
"packageManagers": [
"pip"
],
"cicd": [],
"hasDockerfile": true,
"hasTerraform": false,
"hasKubernetes": false,
"envFiles": [],
"configFiles": []
},
"agents": [
{
"agent": "InjectionTester",
"category": "injection",
"findingCount": 0,
"success": true
},
{
"agent": "AuthBypassAgent",
"category": "auth",
"findingCount": 0,
"success": true
},
{
"agent": "SSRFProber",
"category": "ssrf",
"findingCount": 0,
"success": true
},
{
"agent": "SupplyChainAudit",
"category": "supply-chain",
"findingCount": 0,
"success": true
},
{
"agent": "ConfigAuditor",
"category": "config",
"findingCount": 1,
"success": true
},
{
"agent": "LLMRedTeam",
"category": "llm",
"findingCount": 8,
"success": true
},
{
"agent": "MobileScanner",
"category": "mobile",
"findingCount": 0,
"success": true
},
{
"agent": "GitHistoryScanner",
"category": "history",
"findingCount": 0,
"success": true
},
{
"agent": "CICDScanner",
"category": "cicd",
"findingCount": 0,
"success": true
},
{
"agent": "APIFuzzer",
"category": "api",
"findingCount": 1,
"success": true
}
]
}

212
artifacts/audits/4-third-pass-fix.json Normal file
View File

@@ -0,0 +1,212 @@
{
"score": 92,
"grade": "A",
"gradeLabel": "Ship it!",
"totalFindings": 1,
"totalDepVulns": 0,
"categories": {
"secrets": {
"label": "Secrets",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"injection": {
"label": "Code Vulnerabilities",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"deps": {
"label": "Dependencies",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"auth": {
"label": "Auth & Access Control",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"config": {
"label": "Configuration",
"findingCount": 1,
"deduction": 8,
"counts": {
"critical": 0,
"high": 1,
"medium": 0,
"low": 0
}
},
"supply-chain": {
"label": "Supply Chain",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"api": {
"label": "API Security",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"llm": {
"label": "AI/LLM Security",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
}
},
"findings": [
{
"file": "/code/xf/GIA/Dockerfile",
"line": 26,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
}
],
"depVulns": [],
"remediationPlan": [
{
"priority": 1,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:26",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
}
],
"recon": {
"frameworks": [
"django"
],
"languages": [
"python"
],
"apiRoutes": [
"app/urls.py",
"core/management/commands/backfill_xmpp_attachment_urls.py"
],
"authPatterns": [],
"databases": [],
"cloudProviders": [],
"frontendExposure": [],
"packageManagers": [
"pip"
],
"cicd": [],
"hasDockerfile": true,
"hasTerraform": false,
"hasKubernetes": false,
"envFiles": [],
"configFiles": []
},
"agents": [
{
"agent": "InjectionTester",
"category": "injection",
"findingCount": 0,
"success": true
},
{
"agent": "AuthBypassAgent",
"category": "auth",
"findingCount": 0,
"success": true
},
{
"agent": "SSRFProber",
"category": "ssrf",
"findingCount": 0,
"success": true
},
{
"agent": "SupplyChainAudit",
"category": "supply-chain",
"findingCount": 0,
"success": true
},
{
"agent": "ConfigAuditor",
"category": "config",
"findingCount": 1,
"success": true
},
{
"agent": "LLMRedTeam",
"category": "llm",
"findingCount": 0,
"success": true
},
{
"agent": "MobileScanner",
"category": "mobile",
"findingCount": 0,
"success": true
},
{
"agent": "GitHistoryScanner",
"category": "history",
"findingCount": 0,
"success": true
},
{
"agent": "CICDScanner",
"category": "cicd",
"findingCount": 0,
"success": true
},
{
"agent": "APIFuzzer",
"category": "api",
"findingCount": 0,
"success": true
}
]
}

234
artifacts/audits/5-final-pass-fix.json Normal file
View File

@@ -0,0 +1,234 @@
{
"score": 90,
"grade": "A",
"gradeLabel": "Ship it!",
"totalFindings": 2,
"totalDepVulns": 0,
"categories": {
"secrets": {
"label": "Secrets",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"injection": {
"label": "Code Vulnerabilities",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"deps": {
"label": "Dependencies",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"auth": {
"label": "Auth & Access Control",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"config": {
"label": "Configuration",
"findingCount": 2,
"deduction": 10,
"counts": {
"critical": 0,
"high": 2,
"medium": 0,
"low": 0
}
},
"supply-chain": {
"label": "Supply Chain",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"api": {
"label": "API Security",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
},
"llm": {
"label": "AI/LLM Security",
"findingCount": 0,
"deduction": 0,
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0
}
}
},
"findings": [
{
"file": "/code/xf/GIA/Dockerfile",
"line": 26,
"severity": "high",
"category": "config",
"rule": "DOCKER_RUN_AS_ROOT",
"title": "Docker: Running as Root",
"description": "No USER instruction found. Container runs as root by default.",
"fix": "Add USER nonroot before CMD/ENTRYPOINT",
"cwe": "CWE-250",
"owasp": "A05:2021"
},
{
"file": "/code/xf/GIA/Dockerfile",
"line": 1,
"severity": "high",
"category": "config",
"rule": "DOCKER_NO_USER",
"title": "Dockerfile: No Non-Root USER",
"description": "No USER instruction found. Container runs as root, enabling escape attacks.",
"fix": "Add before CMD: RUN addgroup -S app && adduser -S app -G app\nUSER app",
"cwe": null,
"owasp": null
}
],
"depVulns": [],
"remediationPlan": [
{
"priority": 1,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Docker: Running as Root",
"file": "Dockerfile:26",
"action": "Add USER nonroot before CMD/ENTRYPOINT",
"effort": "low"
},
{
"priority": 2,
"severity": "high",
"category": "config",
"categoryLabel": "CONFIGURATION",
"title": "Dockerfile: No Non-Root USER",
"file": "Dockerfile:1",
"action": "Add before CMD: RUN addgroup -S app && adduser -S app -G app\nUSER app",
"effort": "low"
}
],
"recon": {
"frameworks": [
"django"
],
"languages": [
"python"
],
"apiRoutes": [
"app/urls.py",
"core/management/commands/backfill_xmpp_attachment_urls.py"
],
"authPatterns": [],
"databases": [],
"cloudProviders": [],
"frontendExposure": [],
"packageManagers": [
"pip"
],
"cicd": [],
"hasDockerfile": true,
"hasTerraform": false,
"hasKubernetes": false,
"envFiles": [],
"configFiles": []
},
"agents": [
{
"agent": "InjectionTester",
"category": "injection",
"findingCount": 0,
"success": true
},
{
"agent": "AuthBypassAgent",
"category": "auth",
"findingCount": 0,
"success": true
},
{
"agent": "SSRFProber",
"category": "ssrf",
"findingCount": 0,
"success": true
},
{
"agent": "SupplyChainAudit",
"category": "supply-chain",
"findingCount": 0,
"success": true
},
{
"agent": "ConfigAuditor",
"category": "config",
"findingCount": 2,
"success": true
},
{
"agent": "LLMRedTeam",
"category": "llm",
"findingCount": 0,
"success": true
},
{
"agent": "MobileScanner",
"category": "mobile",
"findingCount": 0,
"success": true
},
{
"agent": "GitHistoryScanner",
"category": "history",
"findingCount": 0,
"success": true
},
{
"agent": "CICDScanner",
"category": "cicd",
"findingCount": 0,
"success": true
},
{
"agent": "APIFuzzer",
"category": "api",
"findingCount": 0,
"success": true
}
]
}

36
artifacts/plans/01-canonical-conversation-events.md
View File

@@ -1,36 +0,0 @@
# Feature Plan: Canonical Conversation Events (Append-Only Timeline)
## Goal
Introduce a canonical append-only event log for conversations so adapters remain stateless and replay/debugging become deterministic.
## Why This Fits GIA
- GIA already has `Person`, `PersonIdentifier`, `ChatSession`, `Message` and multi-transport routing.
- This adds a durable event backbone without replacing current UI/features.
## Scope
- Add `ConversationEvent` model (append-only).
- Event types: `message_created`, `message_edited`, `message_deleted`, `reaction_added`, `reaction_removed`, `read_receipt`, `media_attached`, `participant_added`, `participant_removed`.
- Persist source metadata: transport, upstream IDs, timestamps, actor, payload.
- Write events from transport ingress and internal compose actions.
- Build replay utility for one chat/session.
## Implementation
1. Add model + migration + indexes (`session`, `event_type`, `created_at`, `origin_transport+origin_message_id`).
2. Add write helper in a new module (`core/events/ledger.py`).
3. Update signal/whatsapp/xmpp ingress handlers to emit canonical events.
4. Update compose send/reaction/edit/delete paths to emit canonical events.
5. Add admin/diagnostic read view for event stream by session.
6. Add replay command to regenerate derived projections.
## Acceptance Criteria
- Every new message/reaction/edit/delete creates exactly one canonical event.
- Event ordering is deterministic per session.
- Replay reproduces message projection for a selected session.
- No adapter requires business logic to infer missing event state.
## Risks
- Double-write races during transition.
- Backfill complexity for old messages.
## Out of Scope
- Full migration of all legacy records in one release.

17
artifacts/plans/02-transport-capability-matrix.md
View File

@@ -7,6 +7,22 @@ Define transport feature capabilities centrally so router/policy/UI can make det
- GIA currently spans Signal/WhatsApp/Instagram/XMPP with uneven feature support. - GIA currently spans Signal/WhatsApp/Instagram/XMPP with uneven feature support.
- Prevents silent failures (for example reaction exists internally but cannot be sent outward). - Prevents silent failures (for example reaction exists internally but cannot be sent outward).
## How It Follows Plan 1
- Plan 1 established canonical event flow as the shared source language for transport actions.
- Plan 2 uses that event flow to gate what may be attempted per transport before adapter calls.
- Interlink:
- Canonical events define **what happened** (`reaction_added`, `message_edited`, etc.).
- Capability matrix defines **what is allowed** on each service at execution time.
- Together they prevent drift:
- no silent no-op on unsupported features,
- no adapter-specific policy branching,
- deterministic user-visible failure reasons.
## Required Inputs From Plan 1
- Canonical event types and normalized action shapes are stable.
- Event write path exists for ingress/outbound actions.
- Traceability exists for diagnostics (`trace_id`, source transport metadata).
## Scope ## Scope
- Add capability registry per transport. - Add capability registry per transport.
- Features: reactions, edits, deletes, threaded replies, typing, media classes, read receipts, participant events. - Features: reactions, edits, deletes, threaded replies, typing, media classes, read receipts, participant events.
@@ -23,6 +39,7 @@ Define transport feature capabilities centrally so router/policy/UI can make det
- Unsupported action never calls transport adapter. - Unsupported action never calls transport adapter.
- User receives explicit, actionable error. - User receives explicit, actionable error.
- Service capabilities are test-covered and easy to update. - Service capabilities are test-covered and easy to update.
- Capability decisions are traceable against canonical event/action context.
## Out of Scope ## Out of Scope
- Dynamic remote capability negotiation. - Dynamic remote capability negotiation.

2
artifacts/plans/14-security-audit.md Normal file
View File

@@ -0,0 +1,2 @@
# 14) Run security audit using artifacts/1-initial.json. Generated using ship-safe.
https://github.com/asamassekou10/ship-safe

9
artifacts/plans/14-sensitive-info-hygiene.md
View File

@@ -1,9 +0,0 @@
# 14) Sensitive Information Hygiene
## Goal
Detect and remove sensitive data exposure from code, config, logs, and payload surfaces.
## Minimal Plan
1. Add a repeatable scan for sensitive patterns across repo and runtime-generated artifacts.
2. Expunge discovered sensitive values and replace with safe placeholders or references.
3. Add guardrails to prevent reintroduction and document the remediation workflow.

8
artifacts/plans/15-simplify-task-settings-and-more.md
View File

@@ -10,4 +10,10 @@ This group has no derived tasks yet. To start populating this view:
task settings sound complicated, make them simpler task settings sound complicated, make them simpler
--
# https://gia.zm.is/settings/system/
assume the user cannot access the log
Use a trace id from the dropdown (recent traces), Event Ledger Smoke `sample[].trace_id`, or UR logs.

24
core/clients/signalapi.py
View File

@@ -16,6 +16,22 @@ SIGNAL_UUID_PATTERN = re.compile(
) )
def _safe_parse_send_response(payload_value) -> int | bool:
payload = payload_value
if isinstance(payload_value, str):
try:
payload = orjson.loads(payload_value)
except orjson.JSONDecodeError:
return False
if not isinstance(payload, dict):
return False
try:
ts = payload.get("timestamp")
return int(ts) if ts else False
except (TypeError, ValueError):
return False
def normalize_signal_recipient(recipient: str) -> str: def normalize_signal_recipient(recipient: str) -> str:
raw = str(recipient or "").strip() raw = str(recipient or "").strip()
if not raw: if not raw:
@@ -395,8 +411,8 @@ def send_message_raw_sync(recipient_uuid, text=None, attachments=None):
response.status_code == status.HTTP_201_CREATED response.status_code == status.HTTP_201_CREATED
): # Signal server returns 201 on success ): # Signal server returns 201 on success
try: try:
ts = orjson.loads(response.text).get("timestamp", None) payload = response.json()
return ts if ts else False except ValueError:
except orjson.JSONDecodeError: payload = {}
return False return _safe_parse_send_response(payload)
return False # If response status is not 201 return False # If response status is not 201

72
core/clients/transport.py
View File

@@ -17,6 +17,10 @@ from django.core.cache import cache
from core.clients import signalapi from core.clients import signalapi
from core.messaging import media_bridge from core.messaging import media_bridge
from core.security.attachments import (
validate_attachment_metadata,
validate_attachment_url,
)
from core.transports.capabilities import supports, unsupported_reason from core.transports.capabilities import supports, unsupported_reason
from core.util import logs from core.util import logs
@@ -665,17 +669,21 @@ async def _normalize_gateway_attachment(service: str, row: dict, session):
if isinstance(content, memoryview): if isinstance(content, memoryview):
content = content.tobytes() content = content.tobytes()
if isinstance(content, bytes): if isinstance(content, bytes):
filename, content_type = validate_attachment_metadata(
filename=normalized.get("filename") or "attachment.bin",
content_type=normalized.get("content_type") or "application/octet-stream",
size=normalized.get("size") or len(content),
)
blob_key = media_bridge.put_blob( blob_key = media_bridge.put_blob(
service=service, service=service,
content=content, content=content,
filename=normalized.get("filename") or "attachment.bin", filename=filename,
content_type=normalized.get("content_type") or "application/octet-stream", content_type=content_type,
) )
return { return {
"blob_key": blob_key, "blob_key": blob_key,
"filename": normalized.get("filename") or "attachment.bin", "filename": filename,
"content_type": normalized.get("content_type") "content_type": content_type,
or "application/octet-stream",
"size": normalized.get("size") or len(content), "size": normalized.get("size") or len(content),
} }
@@ -685,33 +693,39 @@ async def _normalize_gateway_attachment(service: str, row: dict, session):
source_url = normalized.get("url") source_url = normalized.get("url")
if source_url: if source_url:
try: try:
async with session.get(source_url) as response: safe_url = validate_attachment_url(source_url)
async with session.get(safe_url) as response:
if response.status == 200: if response.status == 200:
payload = await response.read() payload = await response.read()
blob_key = media_bridge.put_blob( filename, content_type = validate_attachment_metadata(
service=service,
content=payload,
filename=normalized.get("filename") filename=normalized.get("filename")
or source_url.rstrip("/").split("/")[-1] or safe_url.rstrip("/").split("/")[-1]
or "attachment.bin", or "attachment.bin",
content_type=normalized.get("content_type") content_type=normalized.get("content_type")
or response.headers.get( or response.headers.get(
"Content-Type", "application/octet-stream" "Content-Type", "application/octet-stream"
), ),
size=normalized.get("size") or len(payload),
)
blob_key = media_bridge.put_blob(
service=service,
content=payload,
filename=filename,
content_type=content_type,
) )
return { return {
"blob_key": blob_key, "blob_key": blob_key,
"filename": normalized.get("filename") "filename": filename,
or source_url.rstrip("/").split("/")[-1] "content_type": content_type,
or "attachment.bin",
"content_type": normalized.get("content_type")
or response.headers.get(
"Content-Type", "application/octet-stream"
),
"size": normalized.get("size") or len(payload), "size": normalized.get("size") or len(payload),
} }
except Exception: except Exception as exc:
log.warning("%s attachment fetch failed for %s", service, source_url) log.warning(
"%s attachment fetch failed for %s: %s",
service,
source_url,
exc,
)
return normalized return normalized
@@ -1074,21 +1088,27 @@ async def fetch_attachment(service: str, attachment_ref: dict):
if blob_key: if blob_key:
return media_bridge.get_blob(blob_key) return media_bridge.get_blob(blob_key)
if direct_url: if direct_url:
safe_url = validate_attachment_url(direct_url)
timeout = aiohttp.ClientTimeout(total=20) timeout = aiohttp.ClientTimeout(total=20)
async with aiohttp.ClientSession(timeout=timeout) as session: async with aiohttp.ClientSession(timeout=timeout) as session:
async with session.get(direct_url) as response: async with session.get(safe_url) as response:
if response.status != 200: if response.status != 200:
return None return None
content = await response.read() content = await response.read()
return { filename, content_type = validate_attachment_metadata(
"content": content, filename=attachment_ref.get("filename")
"content_type": response.headers.get( or safe_url.rstrip("/").split("/")[-1]
or "attachment.bin",
content_type=response.headers.get(
"Content-Type", "Content-Type",
attachment_ref.get("content_type", "application/octet-stream"), attachment_ref.get("content_type", "application/octet-stream"),
), ),
"filename": attachment_ref.get("filename") size=len(content),
or direct_url.rstrip("/").split("/")[-1] )
or "attachment.bin", return {
"content": content,
"content_type": content_type,
"filename": filename,
"size": len(content), "size": len(content),
} }
return None return None

51
core/clients/whatsapp.py
View File

@@ -17,6 +17,10 @@ from django.core.cache import cache
from core.clients import ClientBase, transport from core.clients import ClientBase, transport
from core.messaging import history, media_bridge, reply_sync from core.messaging import history, media_bridge, reply_sync
from core.models import Message, PersonIdentifier, PlatformChatLink from core.models import Message, PersonIdentifier, PlatformChatLink
from core.security.attachments import (
validate_attachment_metadata,
validate_attachment_url,
)
try: try:
from google.protobuf.json_format import MessageToDict from google.protobuf.json_format import MessageToDict
@@ -3141,31 +3145,42 @@ class WhatsAppClient(ClientBase):
if isinstance(content, memoryview): if isinstance(content, memoryview):
content = content.tobytes() content = content.tobytes()
if isinstance(content, bytes): if isinstance(content, bytes):
filename, content_type = validate_attachment_metadata(
filename=(attachment or {}).get("filename") or "attachment.bin",
content_type=(attachment or {}).get("content_type")
or "application/octet-stream",
size=len(content),
)
return { return {
"content": content, "content": content,
"filename": (attachment or {}).get("filename") or "attachment.bin", "filename": filename,
"content_type": (attachment or {}).get("content_type") "content_type": content_type,
or "application/octet-stream",
"size": len(content), "size": len(content),
} }
url = (attachment or {}).get("url") url = (attachment or {}).get("url")
if url: if url:
safe_url = validate_attachment_url(url)
timeout = aiohttp.ClientTimeout(total=20) timeout = aiohttp.ClientTimeout(total=20)
async with aiohttp.ClientSession(timeout=timeout) as session: async with aiohttp.ClientSession(timeout=timeout) as session:
async with session.get(url) as response: async with session.get(safe_url) as response:
if response.status != 200: if response.status != 200:
return None return None
payload = await response.read() payload = await response.read()
return { filename, content_type = validate_attachment_metadata(
"content": payload, filename=(attachment or {}).get("filename")
"filename": (attachment or {}).get("filename") or safe_url.rstrip("/").split("/")[-1]
or url.rstrip("/").split("/")[-1]
or "attachment.bin", or "attachment.bin",
"content_type": (attachment or {}).get("content_type") content_type=(attachment or {}).get("content_type")
or response.headers.get( or response.headers.get(
"Content-Type", "application/octet-stream" "Content-Type", "application/octet-stream"
), ),
size=len(payload),
)
return {
"content": payload,
"filename": filename,
"content_type": content_type,
"size": len(payload), "size": len(payload),
} }
return None return None
@@ -3320,11 +3335,19 @@ class WhatsAppClient(ClientBase):
payload = await self._fetch_attachment_payload(attachment) payload = await self._fetch_attachment_payload(attachment)
if not payload: if not payload:
continue continue
mime = str(
payload.get("content_type") or "application/octet-stream"
).lower()
data = payload.get("content") or b"" data = payload.get("content") or b""
filename = payload.get("filename") or "attachment.bin" try:
filename, mime = validate_attachment_metadata(
filename=payload.get("filename") or "attachment.bin",
content_type=payload.get("content_type")
or "application/octet-stream",
size=payload.get("size")
or (len(data) if isinstance(data, (bytes, bytearray)) else 0),
)
except Exception as exc:
self.log.warning("whatsapp blocked attachment: %s", exc)
continue
mime = str(mime).lower()
attachment_target = jid_obj if jid_obj is not None else jid attachment_target = jid_obj if jid_obj is not None else jid
send_method = "document" send_method = "document"
if mime.startswith("image/") and hasattr(self._client, "send_image"): if mime.startswith("image/") and hasattr(self._client, "send_image"):
@@ -3372,7 +3395,7 @@ class WhatsAppClient(ClientBase):
sent_ts, sent_ts,
self._normalize_timestamp(self._pluck(response, "Timestamp") or 0), self._normalize_timestamp(self._pluck(response, "Timestamp") or 0),
) )
await _record_bridge(response, sent_ts, body_hint=filename) await _record_bridge(response, sent_ts, body_hint="attachment")
sent_any = True sent_any = True
if getattr(settings, "WHATSAPP_DEBUG", False): if getattr(settings, "WHATSAPP_DEBUG", False):
self.log.debug( self.log.debug(

68
core/clients/xmpp.py
View File

@@ -30,6 +30,10 @@ from core.models import (
User, User,
WorkspaceConversation, WorkspaceConversation,
) )
from core.security.attachments import (
validate_attachment_metadata,
validate_attachment_url,
)
from core.util import logs from core.util import logs
URL_PATTERN = re.compile(r"https?://[^\s<>'\"\\]+") URL_PATTERN = re.compile(r"https?://[^\s<>'\"\\]+")
@@ -49,9 +53,8 @@ def _filename_from_url(url_value):
def _content_type_from_filename_or_url(url_value, default="application/octet-stream"): def _content_type_from_filename_or_url(url_value, default="application/octet-stream"):
filename = _filename_from_url(url_value) _ = url_value
guessed, _ = mimetypes.guess_type(filename) return str(default or "application/octet-stream")
return guessed or default
def _extract_xml_attachment_urls(message_stanza): def _extract_xml_attachment_urls(message_stanza):
@@ -1013,13 +1016,21 @@ class XMPPComponent(ComponentXMPP):
url_value = _clean_url(att.attrib.get("url")) url_value = _clean_url(att.attrib.get("url"))
if not url_value: if not url_value:
continue continue
try:
safe_url = validate_attachment_url(url_value)
filename, content_type = validate_attachment_metadata(
filename=att.attrib.get("filename") or _filename_from_url(safe_url),
content_type=att.attrib.get("content_type")
or "application/octet-stream",
)
except Exception as exc:
self.log.warning("xmpp dropped unsafe attachment url=%s: %s", url_value, exc)
continue
attachments.append( attachments.append(
{ {
"url": url_value, "url": safe_url,
"filename": att.attrib.get("filename") "filename": filename,
or _filename_from_url(url_value), "content_type": content_type,
"content_type": att.attrib.get("content_type")
or "application/octet-stream",
} }
) )
@@ -1028,11 +1039,19 @@ class XMPPComponent(ComponentXMPP):
url_value = _clean_url(oob.text) url_value = _clean_url(oob.text)
if not url_value: if not url_value:
continue continue
guessed_content_type = _content_type_from_filename_or_url(url_value) try:
safe_url = validate_attachment_url(url_value)
filename, guessed_content_type = validate_attachment_metadata(
filename=_filename_from_url(safe_url),
content_type=_content_type_from_filename_or_url(safe_url),
)
except Exception as exc:
self.log.warning("xmpp dropped unsafe oob url=%s: %s", url_value, exc)
continue
attachments.append( attachments.append(
{ {
"url": url_value, "url": safe_url,
"filename": _filename_from_url(url_value), "filename": filename,
"content_type": guessed_content_type, "content_type": guessed_content_type,
} }
) )
@@ -1043,11 +1062,19 @@ class XMPPComponent(ComponentXMPP):
for url_value in extracted_urls: for url_value in extracted_urls:
if url_value in existing_urls: if url_value in existing_urls:
continue continue
guessed_content_type = _content_type_from_filename_or_url(url_value) try:
safe_url = validate_attachment_url(url_value)
filename, guessed_content_type = validate_attachment_metadata(
filename=_filename_from_url(safe_url),
content_type=_content_type_from_filename_or_url(safe_url),
)
except Exception as exc:
self.log.warning("xmpp dropped extracted unsafe url=%s: %s", url_value, exc)
continue
attachments.append( attachments.append(
{ {
"url": url_value, "url": safe_url,
"filename": _filename_from_url(url_value), "filename": filename,
"content_type": guessed_content_type, "content_type": guessed_content_type,
} }
) )
@@ -1397,7 +1424,16 @@ class XMPPComponent(ComponentXMPP):
async def upload_and_send(self, att, upload_slot, recipient_jid, sender_jid): async def upload_and_send(self, att, upload_slot, recipient_jid, sender_jid):
"""Uploads a file and immediately sends the corresponding XMPP message.""" """Uploads a file and immediately sends the corresponding XMPP message."""
upload_url, put_url, auth_header = upload_slot upload_url, put_url, auth_header = upload_slot
headers = {"Content-Type": att["content_type"]} try:
filename, content_type = validate_attachment_metadata(
filename=att.get("filename"),
content_type=att.get("content_type"),
size=att.get("size"),
)
except Exception as exc:
self.log.warning("xmpp blocked outbound attachment: %s", exc)
return None
headers = {"Content-Type": content_type}
if auth_header: if auth_header:
headers["Authorization"] = auth_header headers["Authorization"] = auth_header
@@ -1412,7 +1448,7 @@ class XMPPComponent(ComponentXMPP):
) )
return None return None
self.log.debug( self.log.debug(
"Successfully uploaded %s to %s", att["filename"], upload_url "Successfully uploaded %s to %s", filename, upload_url
) )
# Send XMPP message immediately after successful upload # Send XMPP message immediately after successful upload

2
core/memory/search_backend.py
View File

@@ -140,7 +140,7 @@ class ManticoreMemorySearchBackend(BaseMemorySearchBackend):
def __init__(self): def __init__(self):
self.base_url = str( self.base_url = str(
getattr(settings, "MANTICORE_HTTP_URL", "http://127.0.0.1:9308") getattr(settings, "MANTICORE_HTTP_URL", "http://localhost:9308")
).rstrip("/") ).rstrip("/")
self.table = str( self.table = str(
getattr(settings, "MANTICORE_MEMORY_TABLE", "gia_memory_items") getattr(settings, "MANTICORE_MEMORY_TABLE", "gia_memory_items")

2
core/security/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
"""Security helpers shared across transport adapters."""

169
core/security/attachments.py Normal file
View File

@@ -0,0 +1,169 @@
import ipaddress
import os
import socket
from fnmatch import fnmatch
from urllib.parse import urlparse
from django.conf import settings
from core.util import logs
log = logs.get_logger("security.attachments")
_DEFAULT_ALLOWED_MIME_PATTERNS = (
"image/*",
"video/*",
"audio/*",
"text/plain",
"application/pdf",
"application/zip",
"application/json",
"application/msword",
"application/vnd.*",
)
_DEFAULT_BLOCKED_MIME_TYPES = {
"text/html",
"application/xhtml+xml",
"application/javascript",
"text/javascript",
"application/x-sh",
"application/x-msdownload",
"application/x-dosexec",
"application/x-executable",
}
_BLOCKED_FILENAME_EXTENSIONS = {
".exe",
".dll",
".bat",
".cmd",
".msi",
".sh",
".ps1",
".jar",
".hta",
}
def _strip_content_type(value: str | None) -> str:
raw = str(value or "").strip().lower()
if not raw:
return ""
return raw.split(";", 1)[0].strip()
def normalize_filename(value: str | None, fallback: str = "attachment.bin") -> str:
name = os.path.basename(str(value or "").strip())
name = name.replace("\x00", "").strip()
return name or fallback
def normalized_content_type(
value: str | None,
*,
fallback: str = "application/octet-stream",
) -> str:
clean = _strip_content_type(value)
return clean or fallback
def _allowed_mime_patterns() -> tuple[str, ...]:
configured = getattr(settings, "ATTACHMENT_ALLOWED_MIME_TYPES", None)
if isinstance(configured, (list, tuple)):
patterns = tuple(str(item or "").strip().lower() for item in configured if item)
if patterns:
return patterns
return _DEFAULT_ALLOWED_MIME_PATTERNS
def _blocked_mime_types() -> set[str]:
configured = getattr(settings, "ATTACHMENT_BLOCKED_MIME_TYPES", None)
if isinstance(configured, (list, tuple, set)):
values = {str(item or "").strip().lower() for item in configured if item}
if values:
return values
return set(_DEFAULT_BLOCKED_MIME_TYPES)
def validate_attachment_metadata(
*,
filename: str | None,
content_type: str | None,
size: int | None = None,
) -> tuple[str, str]:
normalized_name = normalize_filename(filename)
normalized_type = normalized_content_type(content_type)
_, ext = os.path.splitext(normalized_name.lower())
if ext in _BLOCKED_FILENAME_EXTENSIONS:
raise ValueError(f"blocked_filename_extension:{ext}")
if normalized_type in _blocked_mime_types():
raise ValueError(f"blocked_mime_type:{normalized_type}")
allow_unmatched = bool(getattr(settings, "ATTACHMENT_ALLOW_UNKNOWN_MIME", False))
if not any(fnmatch(normalized_type, pattern) for pattern in _allowed_mime_patterns()):
if not allow_unmatched:
raise ValueError(f"unsupported_mime_type:{normalized_type}")
max_bytes = int(getattr(settings, "ATTACHMENT_MAX_BYTES", 25 * 1024 * 1024) or 0)
if size and max_bytes > 0 and int(size) > max_bytes:
raise ValueError(f"attachment_too_large:{size}>{max_bytes}")
return normalized_name, normalized_type
def _host_is_private(hostname: str) -> bool:
if not hostname:
return True
lower = hostname.strip().lower()
if lower in {"localhost", "localhost.localdomain"} or lower.endswith(".local"):
return True
try:
addr = ipaddress.ip_address(lower)
return (
addr.is_private
or addr.is_loopback
or addr.is_link_local
or addr.is_multicast
or addr.is_reserved
or addr.is_unspecified
)
except ValueError:
pass
try:
infos = socket.getaddrinfo(lower, None)
except Exception:
return True
for info in infos:
ip = info[4][0]
try:
addr = ipaddress.ip_address(ip)
except ValueError:
return True
if (
addr.is_private
or addr.is_loopback
or addr.is_link_local
or addr.is_multicast
or addr.is_reserved
or addr.is_unspecified
):
return True
return False
def validate_attachment_url(url_value: str | None) -> str:
url_text = str(url_value or "").strip()
parsed = urlparse(url_text)
if parsed.scheme not in {"http", "https"}:
raise ValueError("unsupported_url_scheme")
if not parsed.netloc:
raise ValueError("attachment_url_missing_host")
if parsed.username or parsed.password:
raise ValueError("attachment_url_embedded_credentials")
allow_private = bool(getattr(settings, "ATTACHMENT_ALLOW_PRIVATE_URLS", False))
host = str(parsed.hostname or "").strip()
if not allow_private and _host_is_private(host):
raise ValueError(f"attachment_private_host_blocked:{host or '-'}")
return url_text

36
core/tests/test_attachment_security.py Normal file
View File

@@ -0,0 +1,36 @@
from django.test import SimpleTestCase, override_settings
from core.security.attachments import (
validate_attachment_metadata,
validate_attachment_url,
)
class AttachmentSecurityTests(SimpleTestCase):
def test_blocks_html_payload(self):
with self.assertRaises(ValueError):
validate_attachment_metadata(
filename="payload.html",
content_type="text/html",
size=32,
)
@override_settings(ATTACHMENT_MAX_BYTES=10)
def test_blocks_oversized_payload(self):
with self.assertRaises(ValueError):
validate_attachment_metadata(
filename="dump.bin",
content_type="application/octet-stream",
size=32,
)
def test_blocks_private_url_by_default(self):
with self.assertRaises(ValueError):
validate_attachment_url("http://localhost/internal")
@override_settings(ATTACHMENT_ALLOW_PRIVATE_URLS=True)
def test_allows_private_url_when_explicitly_enabled(self):
self.assertEqual(
"http://localhost/internal",
validate_attachment_url("http://localhost/internal"),
)

93
core/views/osint.py
View File

@@ -3,6 +3,7 @@ from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import date, datetime, timezone as dt_timezone from datetime import date, datetime, timezone as dt_timezone
from decimal import Decimal, InvalidOperation from decimal import Decimal, InvalidOperation
import re
from typing import Any, Callable from typing import Any, Callable
from urllib.parse import urlencode from urllib.parse import urlencode
@@ -20,6 +21,51 @@ from mixins.views import ObjectList
from core.models import Group, Manipulation, Message, Person, PersonIdentifier, Persona from core.models import Group, Manipulation, Message, Person, PersonIdentifier, Persona
_QUERY_MAX_LEN = 400
_QUERY_ALLOWED_PATTERN = re.compile(r"[\w\s@\-\+\.:,#/]+", re.UNICODE)
def _sanitize_search_query(value: str) -> str:
raw = str(value or "").strip()
if not raw:
return ""
trimmed = raw[:_QUERY_MAX_LEN]
cleaned = "".join(_QUERY_ALLOWED_PATTERN.findall(trimmed)).strip()
return cleaned
def _safe_page_number(value: Any) -> int:
try:
page_value = int(value)
except (TypeError, ValueError):
return 1
return max(1, page_value)
def _safe_query_param(request, key: str, default: str = "") -> str:
raw = request.GET.get(key, default)
return str(raw or default).strip()
def _sanitize_query_state(raw: dict[str, Any]) -> dict[str, str]:
cleaned: dict[str, str] = {}
for key, value in (raw or {}).items():
key_text = str(key or "").strip()
if not key_text or len(key_text) > 80:
continue
value_text = str(value or "").strip()
if not value_text:
continue
if key_text in {"q", "query"}:
value_text = _sanitize_search_query(value_text)
elif key_text == "page":
value_text = str(_safe_page_number(value_text))
else:
value_text = value_text[:200]
if value_text:
cleaned[key_text] = value_text
return cleaned
def _context_type(request_type: str) -> str: def _context_type(request_type: str) -> str:
return "modal" if request_type == "page" else request_type return "modal" if request_type == "page" else request_type
@@ -561,12 +607,14 @@ class OSINTListBase(ObjectList):
return lookups return lookups
def _query_dict(self) -> dict[str, Any]: def _query_dict(self) -> dict[str, Any]:
return {k: v for k, v in self.request.GET.items() if v not in {"", None}} return _sanitize_query_state(
{k: v for k, v in self.request.GET.items() if v not in {"", None}}
)
def _apply_list_search( def _apply_list_search(
self, queryset: models.QuerySet, scope: OsintScopeConfig self, queryset: models.QuerySet, scope: OsintScopeConfig
) -> models.QuerySet: ) -> models.QuerySet:
query = self.request.GET.get("q", "").strip() query = _sanitize_search_query(self.request.GET.get("q", ""))
if not query: if not query:
return queryset return queryset
@@ -721,14 +769,16 @@ class OSINTListBase(ObjectList):
} }
if page_obj.has_previous(): if page_obj.has_previous():
previous_page = _safe_page_number(page_obj.previous_page_number())
pagination["previous_url"] = _url_with_query( pagination["previous_url"] = _url_with_query(
list_url, list_url,
_merge_query(query_state, page=page_obj.previous_page_number()), {"page": previous_page},
) )
if page_obj.has_next(): if page_obj.has_next():
next_page = _safe_page_number(page_obj.next_page_number())
pagination["next_url"] = _url_with_query( pagination["next_url"] = _url_with_query(
list_url, list_url,
_merge_query(query_state, page=page_obj.next_page_number()), {"page": next_page},
) )
for entry in page_obj.paginator.get_elided_page_range(page_obj.number): for entry in page_obj.paginator.get_elided_page_range(page_obj.number):
@@ -742,7 +792,7 @@ class OSINTListBase(ObjectList):
"current": entry == page_obj.number, "current": entry == page_obj.number,
"url": _url_with_query( "url": _url_with_query(
list_url, list_url,
_merge_query(query_state, page=entry), {"page": _safe_page_number(entry)},
), ),
} }
) )
@@ -834,7 +884,7 @@ class OSINTSearch(LoginRequiredMixin, View):
S - Size, I - Index, Q - Query, T - Tags, S - Source, R - Ranges, S - Size, I - Index, Q - Query, T - Tags, S - Source, R - Ranges,
S - Sort, S - Sentiment, A - Annotate, D - Dedup, R - Reverse. S - Sort, S - Sentiment, A - Annotate, D - Dedup, R - Reverse.
""" """
query = str(request.GET.get("q") or "").strip() query = _sanitize_search_query(_safe_query_param(request, "q", ""))
tags = tuple( tags = tuple(
token[4:].strip() token[4:].strip()
for token in query.split() for token in query.split()
@@ -845,15 +895,16 @@ class OSINTSearch(LoginRequiredMixin, View):
index=self._scope_key(request.GET.get("scope")), index=self._scope_key(request.GET.get("scope")),
query=query, query=query,
tags=tags, tags=tags,
source=str(request.GET.get("source") or "all").strip().lower() or "all", source=_safe_query_param(request, "source", "all").lower() or "all",
date_from=str(request.GET.get("date_from") or "").strip(), date_from=_safe_query_param(request, "date_from", ""),
date_to=str(request.GET.get("date_to") or "").strip(), date_to=_safe_query_param(request, "date_to", ""),
sort_mode=str(request.GET.get("sort_mode") or "relevance").strip().lower(), sort_mode=_safe_query_param(request, "sort_mode", "relevance").lower(),
sentiment_min=str(request.GET.get("sentiment_min") or "").strip(), sentiment_min=_safe_query_param(request, "sentiment_min", ""),
sentiment_max=str(request.GET.get("sentiment_max") or "").strip(), sentiment_max=_safe_query_param(request, "sentiment_max", ""),
annotate=str(request.GET.get("annotate") or "1").strip() not in {"0", "false", "off"}, annotate=_safe_query_param(request, "annotate", "1")
dedup=str(request.GET.get("dedup") or "").strip() in {"1", "true", "on"}, not in {"0", "false", "off"},
reverse=str(request.GET.get("reverse") or "").strip() in {"1", "true", "on"}, dedup=_safe_query_param(request, "dedup", "") in {"1", "true", "on"},
reverse=_safe_query_param(request, "reverse", "") in {"1", "true", "on"},
) )
def _parse_date_boundaries(self, plan: "OSINTSearch.SearchPlan") -> tuple[datetime | None, datetime | None]: def _parse_date_boundaries(self, plan: "OSINTSearch.SearchPlan") -> tuple[datetime | None, datetime | None]:
@@ -1069,7 +1120,9 @@ class OSINTSearch(LoginRequiredMixin, View):
return "all" return "all"
def _query_state(self, request) -> dict[str, Any]: def _query_state(self, request) -> dict[str, Any]:
return {k: v for k, v in request.GET.items() if v not in {None, ""}} return _sanitize_query_state(
{k: v for k, v in request.GET.items() if v not in {None, ""}}
)
def _apply_common_filters( def _apply_common_filters(
self, self,
@@ -1359,14 +1412,16 @@ class OSINTSearch(LoginRequiredMixin, View):
} }
if page_obj.has_previous(): if page_obj.has_previous():
previous_page = _safe_page_number(page_obj.previous_page_number())
pagination["previous_url"] = _url_with_query( pagination["previous_url"] = _url_with_query(
list_url, list_url,
_merge_query(query_state, page=page_obj.previous_page_number()), {"page": previous_page},
) )
if page_obj.has_next(): if page_obj.has_next():
next_page = _safe_page_number(page_obj.next_page_number())
pagination["next_url"] = _url_with_query( pagination["next_url"] = _url_with_query(
list_url, list_url,
_merge_query(query_state, page=page_obj.next_page_number()), {"page": next_page},
) )
for entry in page_obj.paginator.get_elided_page_range(page_obj.number): for entry in page_obj.paginator.get_elided_page_range(page_obj.number):
@@ -1380,7 +1435,7 @@ class OSINTSearch(LoginRequiredMixin, View):
"current": entry == page_obj.number, "current": entry == page_obj.number,
"url": _url_with_query( "url": _url_with_query(
list_url, list_url,
_merge_query(query_state, page=entry), {"page": _safe_page_number(entry)},
), ),
} }
) )

55
core/views/signal.py
View File

@@ -17,6 +17,28 @@ from core.presence import latest_state_for_people
from core.views.manage.permissions import SuperUserRequiredMixin from core.views.manage.permissions import SuperUserRequiredMixin
def _safe_json_list(text_value):
try:
payload = orjson.loads(text_value)
except orjson.JSONDecodeError:
return []
return payload if isinstance(payload, list) else []
def _sanitize_signal_rows(rows):
safe_rows = []
for row in rows:
if not isinstance(row, dict):
continue
safe_row = {}
for key, value in row.items():
if isinstance(key, str) and len(key) <= 100:
if isinstance(value, (str, int, float, bool)) or value is None:
safe_row[key] = value
safe_rows.append(safe_row)
return safe_rows
class CustomObjectRead(ObjectRead): class CustomObjectRead(ObjectRead):
def post(self, request, *args, **kwargs): def post(self, request, *args, **kwargs):
self.request = request self.request = request
@@ -171,21 +193,28 @@ class SignalContactsList(SuperUserRequiredMixin, ObjectList):
list_url_args = ["type", "pk"] list_url_args = ["type", "pk"]
def get_queryset(self, *args, **kwargs): def get_queryset(self, *args, **kwargs):
# url = signal:8080/v1/accounts
# /v1/configuration/{number}/settings
# /v1/identities/{number}
# /v1/contacts/{number}
# response = requests.get(
# f"http://signal:8080/v1/configuration/{self.kwargs['pk']}/settings"
# )
# config = orjson.loads(response.text)
base = getattr(settings, "SIGNAL_HTTP_URL", "http://signal:8080").rstrip("/") base = getattr(settings, "SIGNAL_HTTP_URL", "http://signal:8080").rstrip("/")
response = requests.get(f"{base}/v1/identities/{self.kwargs['pk']}") try:
identities = orjson.loads(response.text) response = requests.get(
f"{base}/v1/identities/{self.kwargs['pk']}", timeout=15
)
response.raise_for_status()
identities = _sanitize_signal_rows(response.json() or [])
except requests.RequestException:
identities = []
except ValueError:
identities = []
response = requests.get(f"{base}/v1/contacts/{self.kwargs['pk']}") try:
contacts = orjson.loads(response.text) response = requests.get(
f"{base}/v1/contacts/{self.kwargs['pk']}", timeout=15
)
response.raise_for_status()
contacts = _sanitize_signal_rows(response.json() or [])
except requests.RequestException:
contacts = []
except ValueError:
contacts = []
# add identities to contacts # add identities to contacts
for contact in contacts: for contact in contacts:

0
mixins/__init__.py Normal file
View File

0
mixins/__main__.py Normal file
View File

100
mixins/restrictions.py Normal file
View File

@@ -0,0 +1,100 @@
from django.core.exceptions import FieldDoesNotExist, ImproperlyConfigured
from django.core.paginator import Paginator
from django.db.models import QuerySet
class RestrictedViewMixin:
"""
This mixin overrides two helpers in order to pass the user object to the filters.
get_queryset alters the objects returned for list views.
get_form_kwargs passes the request object to the form class. Remaining permissions
checks are in forms.py
"""
allow_empty = True
queryset = None
model = None
paginate_by = None
paginate_orphans = 0
context_object_name = None
paginator_class = Paginator
page_kwarg = "page"
ordering = None
def set_extra_args(self, user):
"""
This function is overriden to filter the objects by the requesting user.
"""
self.extra_permission_args = {}
def get_queryset(self, **kwargs):
"""
This function is overriden to filter the objects by the requesting user.
"""
self.set_extra_args(self.request.user)
if self.queryset is not None:
queryset = self.queryset
if isinstance(queryset, QuerySet):
# queryset = queryset.all()
queryset = queryset.filter(
user=self.request.user, **self.extra_permission_args
)
elif self.model is not None:
queryset = self.model._default_manager.filter(
user=self.request.user, **self.extra_permission_args
)
else:
raise ImproperlyConfigured(
"%(cls)s is missing a QuerySet. Define "
"%(cls)s.model, %(cls)s.queryset, or override "
"%(cls)s.get_queryset()." % {"cls": self.__class__.__name__}
)
if hasattr(self, "get_ordering"):
ordering = self.get_ordering()
if ordering:
if isinstance(ordering, str):
ordering = (ordering,)
queryset = queryset.order_by(*ordering)
return queryset
def get_form_kwargs(self):
"""Passes the request object to the form class.
This is necessary to only display members that belong to a given user"""
kwargs = super().get_form_kwargs()
kwargs["request"] = self.request
return kwargs
class RestrictedFormMixin:
"""
This mixin is used to restrict the queryset of a form to the current user.
The request object is passed from the view.
Fieldargs is used to pass additional arguments to the queryset filter.
"""
fieldargs = {}
# TODO: implement set_extra_args to check more permissions here
# for completeness, however as views open forms, the permissions
# are already checked there, so it may not be necessary.
def __init__(self, *args, **kwargs):
# self.fieldargs = {}
self.request = kwargs.pop("request")
super().__init__(*args, **kwargs)
for field in self.fields:
# Check it's not something like a CharField which has no queryset
if not hasattr(self.fields[field], "queryset"):
continue
model = self.fields[field].queryset.model
# Check if the model has a user field
try:
model._meta.get_field("user")
# Add the user to the queryset filters
self.fields[field].queryset = model.objects.filter(
user=self.request.user, **self.fieldargs.get(field, {})
)
except FieldDoesNotExist:
pass

1
mixins/templates/mixins/partials/close-modal.html Normal file
View File

@@ -0,0 +1 @@
<button class="modal-close is-large" aria-label="close"></button>

3
mixins/templates/mixins/partials/close-widget.html Normal file
View File

@@ -0,0 +1,3 @@
<i
class="fa-solid fa-xmark has-text-grey-light float-right"
onclick='grid.removeWidget("widget-{{ unique }}");grid.compact();'></i>

3
mixins/templates/mixins/partials/close-window.html Normal file
View File

@@ -0,0 +1,3 @@
<i
class="fa-solid fa-xmark has-text-grey-light float-right"
data-script="on click remove the closest <nav/>"></i>

77
mixins/templates/mixins/partials/generic-detail.html Normal file
View File

@@ -0,0 +1,77 @@
{% load pretty %}
{% load cache %}
{% cache 600 generic_detail request.user.id object live %}
{% include 'mixins/partials/notify.html' %}
{% if live is not None %}
<h1 class="title">Live {{ context_object_name_singular }} info</h1>
<table class="table is-fullwidth is-hoverable">
<thead>
<th>attribute</th>
<th>value</th>
</thead>
<tbody>
{% block live_tbody %}
{% for key, item in live.items %}
{% if key in pretty %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
<pre>{{ item|pretty }}</pre>
{% endif %}
</td>
</tr>
{% else %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
{{ item }}
{% endif %}
</td>
</tr>
{% endif %}
{% endfor %}
{% endblock %}
</tbody>
</table>
{% endif %}
{% if object is not None %}
<h1 class="title">{{ title_singular }} info</h1>
<table class="table is-fullwidth is-hoverable">
<thead>
<th>attribute</th>
<th>value</th>
</thead>
<tbody>
{% block tbody %}
{% for key, item in object.items %}
{% if key in pretty %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
<pre>{{ item|pretty }}</pre>
{% endif %}
</td>
</tr>
{% else %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
{{ item }}
{% endif %}
</td>
</tr>
{% endif %}
{% endfor %}
{% endblock %}
</tbody>
</table>
{% endif %}
{% endcache %}

7
mixins/templates/mixins/partials/notify.html Normal file
View File

@@ -0,0 +1,7 @@
<div id="notification">
{% if message is not None %}
<div class="notification is-{{ class }}" hx-ext="remove-me" remove-me="3s">
{{ message }}
</div>
{% endif %}
</div>

26
mixins/templates/mixins/window-content/object-form.html Normal file
View File

@@ -0,0 +1,26 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
{% load crispy_forms_tags %}
{% load crispy_forms_bulma_field %}
<form
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-post="{{ submit_url }}"
hx-target="#modals-here"
hx-swap="innerHTML">
{% csrf_token %}
{{ form|crispy }}
{% if hide_cancel is not True %}
<button
type="button"
class="button is-light modal-close-button">
Cancel
</button>
{% endif %}
<button type="submit" class="button modal-close-button">Submit</button>
</form>

61
mixins/templates/mixins/window-content/object.html Normal file
View File

@@ -0,0 +1,61 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
<div class="buttons">
{% if submit_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-get="{{ submit_url }}"
hx-trigger="click"
hx-target="#modals-here"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-plus"></i>
</span>
<span>{{ title_singular }}</span>
</span>
</button>
{% endif %}
{% if delete_all_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-delete="{{ delete_all_url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
hx-confirm="Are you sure you wish to delete all {{ context_object_name }}?"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-xmark"></i>
</span>
<span>Delete all {{ context_object_name }} </span>
</span>
</button>
{% endif %}
{% for button in extra_buttons %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-{{ button.method }}="{{ button.url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
{% if button.confirm %}hx-confirm="Are you sure you wish to {{ button.action }}?"{% endif %}
class="button">
<span class="icon-text">
<span class="icon">
<i class="{{ button.icon }}"></i>
</span>
<span>{{ button.label }}</span>
</span>
</button>
{% endfor %}
</div>
{% include detail_template %}

61
mixins/templates/mixins/window-content/objects.html Normal file
View File

@@ -0,0 +1,61 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
<div class="buttons">
{% if submit_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-get="{{ submit_url }}"
hx-trigger="click"
hx-target="#modals-here"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-plus"></i>
</span>
<span>{{ title_singular }}</span>
</span>
</button>
{% endif %}
{% if delete_all_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-delete="{{ delete_all_url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
hx-confirm="Are you sure you wish to delete all {{ context_object_name }}?"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-xmark"></i>
</span>
<span>Delete all {{ context_object_name }} </span>
</span>
</button>
{% endif %}
{% for button in extra_buttons %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-{{ button.method }}="{{ button.url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
{% if button.confirm %}hx-confirm="Are you sure you wish to {{ button.action }}?"{% endif %}
class="button">
<span class="icon-text">
<span class="icon">
<i class="{{ button.icon }}"></i>
</span>
<span>{{ button.label }}</span>
</span>
</button>
{% endfor %}
</div>
{% include list_template %}

20
mixins/templates/mixins/wm/modal.html Normal file
View File

@@ -0,0 +1,20 @@
{% load static %}
<script src="{% static 'modal.js' %}"></script>
{% block scripts %}
{% endblock %}
{% block styles %}
{% endblock %}
<div id="modal" class="modal is-active is-clipped">
<div class="modal-background"></div>
<div class="modal-content">
<div class="box">
{% block modal_content %}
{% include window_content %}
{% endblock %}
{% include 'mixins/partials/close-modal.html' %}
</div>
</div>
</div>

6
mixins/templates/mixins/wm/page.html Normal file
View File

@@ -0,0 +1,6 @@
{% extends "base.html" %}
{% block content %}
{% include window_content %}
{% endblock %}

17
mixins/templates/mixins/wm/panel.html Normal file
View File

@@ -0,0 +1,17 @@
<nav class="panel">
<p class="panel-heading" style="padding: .2em; line-height: .5em;">
<i class="fa-solid fa-arrows-up-down-left-right has-text-grey-light"></i>
{% block close_button %}
{% include 'mixins/partials/close-window.html' %}
{% endblock %}
{% block heading %}
{% endblock %}
</p>
<article class="panel-block is-active">
<div class="control">
{% block panel_content %}
{% endblock %}
</div>
</article>
</nav>

37
mixins/templates/mixins/wm/widget.html Normal file
View File

@@ -0,0 +1,37 @@
<div id="widget">
<div id="widget-{{ unique }}" class="grid-stack-item" {% block widget_options %}{% if widget_options is None %}gs-w="6" gs-h="1" gs-y="20" gs-x="0"{% else %}{% autoescape off %}{{ widget_options }}{% endautoescape %}{% endif %}{% endblock %}>
<div class="grid-stack-item-content">
<nav class="panel">
<p class="panel-heading" style="padding: .2em; line-height: .5em;">
<i class="fa-solid fa-arrows-up-down-left-right has-text-grey-light"></i>
{% block close_button %}
{% include 'mixins/partials/close-widget.html' %}
{% endblock %}
<i
class="fa-solid fa-arrows-minimize has-text-grey-light float-right"
onclick='grid.compact();'></i>
{% block heading %}
{{ title }}
{% endblock %}
</p>
<article class="panel-block is-active">
<div class="control">
{% block panel_content %}
{% include window_content %}
{% endblock %}
</div>
</article>
</nav>
</div>
</div>
</div>
<script>
{% block custom_script %}
{% endblock %}
var widget_event = new Event('load-widget');
document.dispatchEvent(widget_event);
</script>
{% block custom_end %}
{% endblock %}

10
mixins/templates/mixins/wm/window.html Normal file
View File

@@ -0,0 +1,10 @@
<magnet-block attract-distance="10" align-to="outer|center" class="floating-window">
{% extends 'mixins/wm/panel.html' %}
{% block heading %}
{{ title }}
{% endblock %}
{% block panel_content %}
{% include window_content %}
{% endblock %}
</magnet-block>

466
mixins/views.py Normal file
View File

@@ -0,0 +1,466 @@
import uuid
from django.http import Http404, HttpResponse, HttpResponseBadRequest
from django.urls import reverse
from django.views.generic.detail import DetailView
from django.views.generic.edit import CreateView, DeleteView, UpdateView
from django.views.generic.list import ListView
from rest_framework.parsers import FormParser
from mixins.restrictions import RestrictedViewMixin
class AbortSave(Exception):
pass
class ObjectNameMixin(object):
def __init__(self, *args, **kwargs):
if self.model is None:
self.title = self.context_object_name.title()
self.title_singular = self.context_object_name_singular.title()
else:
self.title_singular = self.model._meta.verbose_name.title() # Hook
self.context_object_name_singular = self.title_singular.lower() # hook
self.title = self.model._meta.verbose_name_plural.title() # Hooks
self.context_object_name = self.title.lower() # hooks
self.context_object_name = self.context_object_name.replace(" ", "")
self.context_object_name_singular = (
self.context_object_name_singular.replace(" ", "")
)
super().__init__(*args, **kwargs)
class ObjectList(RestrictedViewMixin, ObjectNameMixin, ListView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/objects.html"
list_template = None
page_title = None
page_subtitle = None
list_url_name = None
# WARNING: TAKEN FROM locals()
list_url_args = ["type"]
submit_url_name = None
submit_url_args = ["type"]
delete_all_url_name = None
widget_options = None
extra_buttons = None
def queryset_mutate(self, queryset):
pass
# copied from BaseListView
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.request = request
self.object_list = self.get_queryset(**kwargs)
if isinstance(self.object_list, HttpResponse):
return self.object_list
if isinstance(self.object_list, HttpResponseBadRequest):
return self.object_list
self.queryset_mutate(self.object_list)
allow_empty = self.get_allow_empty()
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
list_url_args = {}
for arg in self.list_url_args:
if arg in locals():
list_url_args[arg] = locals()[arg]
elif arg in kwargs:
list_url_args[arg] = kwargs[arg]
orig_type = type
if type == "page":
type = "modal"
if not allow_empty:
# When pagination is enabled and object_list is a queryset,
# it's better to do a cheap query than to load the unpaginated
# queryset in memory.
if self.get_paginate_by(self.object_list) is not None and hasattr(
self.object_list, "exists"
):
is_empty = not self.object_list.exists()
else:
is_empty = not self.object_list
if is_empty:
raise Http404("Empty list")
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
context = self.get_context_data()
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["list_template"] = self.list_template
context["page_title"] = self.page_title
context["page_subtitle"] = self.page_subtitle
context["type"] = type
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
if self.submit_url_name is not None:
context["submit_url"] = reverse(
self.submit_url_name, kwargs=submit_url_args
)
if self.list_url_name is not None:
context["list_url"] = reverse(self.list_url_name, kwargs=list_url_args)
if self.delete_all_url_name:
context["delete_all_url"] = reverse(self.delete_all_url_name)
if self.widget_options:
context["widget_options"] = self.widget_options
if self.extra_buttons is not None:
context["extra_buttons"] = self.extra_buttons
# Return partials for HTMX
if self.request.htmx:
if request.headers["HX-Target"] == self.context_object_name + "-table":
self.template_name = self.list_template
elif orig_type == "page":
self.template_name = self.list_template
else:
context["window_content"] = self.list_template
return self.render_to_response(context)
def post(self, request, *args, **kwargs):
self.request = request
return super().get(request, *args, **kwargs)
class ObjectCreate(RestrictedViewMixin, ObjectNameMixin, CreateView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object-form.html"
parser_classes = [FormParser]
page_title = None
page_subtitle = None
model = None
submit_url_name = None
submit_url_args = ["type"]
request = None
# Whether to hide the cancel button in the form
hide_cancel = False
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.title = "Create " + self.context_object_name_singular
def post_save(self, obj):
pass
def pre_save_mutate(self, user, obj):
pass
def form_valid(self, form):
obj = form.save(commit=False)
if self.request is None:
raise Exception("Request is None")
obj.user = self.request.user
try:
self.pre_save_mutate(self.request.user, obj)
except AbortSave as e:
context = {"message": f"Failed to save: {e}", "class": "danger"}
return self.render_to_response(context)
obj.save()
form.save_m2m()
self.post_save(obj)
context = {"message": "Object created", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def form_invalid(self, form):
"""If the form is invalid, render the invalid form."""
return self.get(self.request, **self.kwargs, form=form)
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
self.request = request
self.kwargs = kwargs
if type == "widget":
self.hide_cancel = True
if type == "page":
type = "modal"
self.object = None
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
submit_url = reverse(self.submit_url_name, kwargs=submit_url_args)
context = self.get_context_data()
form = kwargs.get("form", None)
if form:
context["form"] = form
context["unique"] = unique
context["window_content"] = self.window_content
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
context["submit_url"] = submit_url
context["type"] = type
context["hide_cancel"] = self.hide_cancel
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
response = self.render_to_response(context)
# response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def post(self, request, *args, **kwargs):
self.request = request
self.template_name = "mixins/partials/notify.html"
return super().post(request, *args, **kwargs)
class ObjectRead(RestrictedViewMixin, ObjectNameMixin, DetailView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object.html"
detail_template = "mixins/partials/generic-detail.html"
page_title = None
page_subtitle = None
model = None
# submit_url_name = None
detail_url_name = None
# WARNING: TAKEN FROM locals()
detail_url_args = ["type"]
request = None
extra_buttons = None
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest()
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
detail_url_args = {}
for arg in self.detail_url_args:
if arg in locals():
detail_url_args[arg] = locals()[arg]
elif arg in kwargs:
detail_url_args[arg] = kwargs[arg]
self.request = request
self.kwargs = kwargs
self.object = self.get_object(**kwargs)
if isinstance(self.object, HttpResponse):
return self.object
orig_type = type
if type == "page":
type = "modal"
context = self.get_context_data()
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["detail_template"] = self.detail_template
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
context["type"] = type
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
if self.detail_url_name is not None:
context["detail_url"] = reverse(
self.detail_url_name, kwargs=detail_url_args
)
if self.extra_buttons is not None:
context["extra_buttons"] = self.extra_buttons
# Return partials for HTMX
if self.request.htmx:
if request.headers["HX-Target"] == self.context_object_name + "-info":
self.template_name = self.detail_template
elif orig_type == "page":
self.template_name = self.detail_template
else:
context["window_content"] = self.detail_template
return self.render_to_response(context)
def post(self, request, *args, **kwargs):
self.request = request
return super().get(request, *args, **kwargs)
class ObjectUpdate(RestrictedViewMixin, ObjectNameMixin, UpdateView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object-form.html"
parser_classes = [FormParser]
page_title = None
page_subtitle = None
model = None
submit_url_name = None
submit_url_args = ["type", "pk"]
request = None
# Whether pk is required in the get request
pk_required = True
# Whether to hide the cancel button in the form
hide_cancel = False
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.title = "Update " + self.context_object_name_singular
def post_save(self, obj):
pass
def form_valid(self, form):
obj = form.save(commit=False)
if self.request is None:
raise Exception("Request is None")
obj.save()
form.save_m2m()
self.post_save(obj)
context = {"message": "Object updated", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def form_invalid(self, form):
"""If the form is invalid, render the invalid form."""
return self.get(self.request, **self.kwargs, form=form)
def get(self, request, *args, **kwargs):
self.request = request
type = kwargs.get("type", None)
pk = kwargs.get("pk", None)
if not type:
return HttpResponseBadRequest("No type specified")
if not pk:
if self.pk_required:
return HttpResponseBadRequest("No pk specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
if type == "widget":
self.hide_cancel = True
if type == "page":
type = "modal"
self.object = self.get_object()
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
submit_url = reverse(self.submit_url_name, kwargs=submit_url_args)
context = self.get_context_data()
form = kwargs.get("form", None)
if form:
context["form"] = form
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
context["submit_url"] = submit_url
context["type"] = type
context["hide_cancel"] = self.hide_cancel
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
response = self.render_to_response(context)
# response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def post(self, request, *args, **kwargs):
self.request = request
self.template_name = "mixins/partials/notify.html"
return super().post(request, *args, **kwargs)
class ObjectDelete(RestrictedViewMixin, ObjectNameMixin, DeleteView):
model = None
template_name = "mixins/partials/notify.html"
# Overriden to prevent success URL from being used
def delete(self, request, *args, **kwargs):
"""
Call the delete() method on the fetched object and then redirect to the
success URL.
"""
self.object = self.get_object()
# success_url = self.get_success_url()
self.object.delete()
context = {"message": "Object deleted", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
# This will be used in newer Django versions, until then we get a warning
def form_valid(self, form):
"""
Call the delete() method on the fetched object.
"""
self.object = self.get_object()
self.object.delete()
context = {"message": "Object deleted", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response

74
requirements.txt
View File

@@ -1,41 +1,35 @@
wheel wheel==0.45.1
uwsgi uwsgi==2.0.28
django django==4.2.19
pre-commit pre-commit==4.2.0
django-crispy-forms==1.14.0 django-crispy-forms==1.14.0
crispy-bulma crispy-bulma==0.11.0
# manticoresearch djangorestframework==3.15.2
# stripe uvloop==0.21.0
django-rest-framework django-htmx==1.21.0
uvloop cryptography==44.0.2
django-htmx django-debug-toolbar==4.4.6
cryptography django-debug-toolbar-template-profiler==2.1.0
django-debug-toolbar orjson==3.10.15
django-debug-toolbar-template-profiler msgpack==1.1.0
orjson apscheduler==3.10.4
msgpack watchfiles==1.0.5
apscheduler django-otp==1.6.0
watchfiles django-two-factor-auth==1.17.0
django-otp django-otp-yubikey==1.1.0
django-two-factor-auth phonenumbers==8.13.55
django-otp-yubikey qrcode==8.0
phonenumbers pydantic==2.10.6
qrcode redis==6.2.0
pydantic hiredis==3.1.0
# glom django-cachalot==2.7.0
git+https://git.zm.is/XF/django-crud-mixins django-redis==5.4.0
# pyroscope-io requests==2.32.3
# For caching signalbot==0.16.0
redis<7 openai==1.66.3
hiredis aiograpi==0.0.4
django-cachalot aiomysql==0.2.0
django_redis slixmpp==1.10.0
requests neonize==0.3.12
signalbot watchdog==6.0.0
openai uvicorn==0.34.0
aiograpi
aiomysql
slixmpp
neonize
watchdog
uvicorn

6
scripts/quadlet/manage.sh
View File

@@ -37,6 +37,7 @@ PROSODY_CERTS_DIR="${QUADLET_PROSODY_CERTS_DIR:-$ROOT_DIR/.podman/gia_prosody_ce
PROSODY_DATA_DIR="${QUADLET_PROSODY_DATA_DIR:-$ROOT_DIR/.podman/gia_prosody_data}" PROSODY_DATA_DIR="${QUADLET_PROSODY_DATA_DIR:-$ROOT_DIR/.podman/gia_prosody_data}"
PROSODY_LOGS_DIR="${QUADLET_PROSODY_LOGS_DIR:-$ROOT_DIR/.podman/gia_prosody_logs}" PROSODY_LOGS_DIR="${QUADLET_PROSODY_LOGS_DIR:-$ROOT_DIR/.podman/gia_prosody_logs}"
PROSODY_ENABLED="${PROSODY_ENABLED:-false}" PROSODY_ENABLED="${PROSODY_ENABLED:-false}"
ENSURE_XMPP_SECRET_SCRIPT="$ROOT_DIR/utilities/prosody/ensure_xmpp_secret.sh"
if [[ -n "${STACK_ID}" ]]; then if [[ -n "${STACK_ID}" ]]; then
VRUN_DIR="/code/vrun/${STACK_ID}" VRUN_DIR="/code/vrun/${STACK_ID}"
else else
@@ -48,6 +49,10 @@ load_env() {
. "$STACK_ENV" . "$STACK_ENV"
set +a set +a
PROSODY_ENABLED="${PROSODY_ENABLED:-false}" PROSODY_ENABLED="${PROSODY_ENABLED:-false}"
if [[ -x "$ENSURE_XMPP_SECRET_SCRIPT" ]]; then
XMPP_SECRET="$("$ENSURE_XMPP_SECRET_SCRIPT" "$STACK_ENV")"
export XMPP_SECRET
fi
} }
is_remote() { is_remote() {
@@ -226,6 +231,7 @@ start_stack() {
--replace \ --replace \
--name "$PROSODY_CONTAINER" \ --name "$PROSODY_CONTAINER" \
--pod "$POD_NAME" \ --pod "$POD_NAME" \
--env-file "$STACK_ENV" \
-v "$PROSODY_CONFIG_FILE:/etc/prosody/prosody.cfg.lua:ro" \ -v "$PROSODY_CONFIG_FILE:/etc/prosody/prosody.cfg.lua:ro" \
-v "$PROSODY_CERTS_DIR:/etc/prosody/certs" \ -v "$PROSODY_CERTS_DIR:/etc/prosody/certs" \
-v "$PROSODY_DATA_DIR:/var/lib/prosody" \ -v "$PROSODY_DATA_DIR:/var/lib/prosody" \

2
scripts/quadlet/render_units.py
View File

@@ -184,7 +184,7 @@ WantedBy={target_ref}
f"Pod={pod_ref}", f"Pod={pod_ref}",
f"User={host_uid}:{host_gid}", f"User={host_uid}:{host_gid}",
f"EnvironmentFile={env_file}", f"EnvironmentFile={env_file}",
"Environment=SIGNAL_HTTP_URL=http://127.0.0.1:8080", "Environment=SIGNAL_HTTP_URL=http://localhost:8080",
f"Environment=APP_DATABASE_PATH={app_db_path_in_container}", f"Environment=APP_DATABASE_PATH={app_db_path_in_container}",
f"Volume={repo_dir}:/code", f"Volume={repo_dir}:/code",
f"Volume={sqlite_data_dir}:/conf", f"Volume={sqlite_data_dir}:/conf",

6
stack.env.example
View File

@@ -31,6 +31,7 @@ EVENT_PRIMARY_WRITE_PATH=false
XMPP_ADDRESS=127.0.0.1 XMPP_ADDRESS=127.0.0.1
XMPP_JID=jews.example.com XMPP_JID=jews.example.com
XMPP_PORT=8888 XMPP_PORT=8888
# Auto-generated if empty by Prosody startup helpers.
XMPP_SECRET= XMPP_SECRET=
# Optional Prosody container storage/config paths used by utilities/prosody/manage_prosody_container.sh # Optional Prosody container storage/config paths used by utilities/prosody/manage_prosody_container.sh
@@ -41,6 +42,9 @@ QUADLET_PROSODY_LOGS_DIR=./.podman/gia_prosody_logs
# Memory/wiki search backend foundation # Memory/wiki search backend foundation
MEMORY_SEARCH_BACKEND=django MEMORY_SEARCH_BACKEND=django
MANTICORE_HTTP_URL=http://127.0.0.1:9308 MANTICORE_HTTP_URL=http://localhost:9308
MANTICORE_MEMORY_TABLE=gia_memory_items MANTICORE_MEMORY_TABLE=gia_memory_items
MANTICORE_HTTP_TIMEOUT=5 MANTICORE_HTTP_TIMEOUT=5
ATTACHMENT_MAX_BYTES=26214400
ATTACHMENT_ALLOW_PRIVATE_URLS=false
ATTACHMENT_ALLOW_UNKNOWN_MIME=false

58
utilities/prosody/ensure_xmpp_secret.sh Executable file
View File

@@ -0,0 +1,58 @@
#!/usr/bin/env bash
set -euo pipefail
STACK_ENV="${1:-}"
if [[ -z "$STACK_ENV" ]]; then
echo "Usage: $0 /path/to/stack.env" >&2
exit 2
fi
mkdir -p "$(dirname "$STACK_ENV")"
touch "$STACK_ENV"
current_secret=""
if grep -q '^XMPP_SECRET=' "$STACK_ENV"; then
current_secret="$(grep '^XMPP_SECRET=' "$STACK_ENV" | head -n1 | cut -d= -f2- | tr -d '"' | tr -d "'" | tr -d '\r' | tr -d '\n')"
fi
if [[ -n "$current_secret" ]]; then
printf "%s" "$current_secret"
exit 0
fi
generate_secret() {
if command -v openssl >/dev/null 2>&1; then
openssl rand -base64 48 | tr -d '\n'
return 0
fi
if command -v python3 >/dev/null 2>&1; then
python3 -c 'import secrets; print(secrets.token_urlsafe(48))'
return 0
fi
head -c 48 /dev/urandom | base64 | tr -d '\n'
}
secret="$(generate_secret)"
if [[ -z "$secret" ]]; then
echo "Failed to generate XMPP_SECRET." >&2
exit 1
fi
tmp="$(mktemp)"
awk -v s="$secret" '
BEGIN { done = 0 }
/^XMPP_SECRET=/ {
if (!done) {
print "XMPP_SECRET=" s
done = 1
}
next
}
{ print }
END {
if (!done) print "XMPP_SECRET=" s
}
' "$STACK_ENV" > "$tmp"
mv "$tmp" "$STACK_ENV"
printf "%s" "$secret"

32
utilities/prosody/manage_prosody_container.sh
View File

@@ -3,6 +3,7 @@ set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
STACK_ENV="${STACK_ENV:-$ROOT_DIR/stack.env}" STACK_ENV="${STACK_ENV:-$ROOT_DIR/stack.env}"
ENSURE_XMPP_SECRET_SCRIPT="$ROOT_DIR/utilities/prosody/ensure_xmpp_secret.sh"
if [[ -f "$STACK_ENV" ]]; then if [[ -f "$STACK_ENV" ]]; then
set -a set -a
@@ -10,6 +11,11 @@ if [[ -f "$STACK_ENV" ]]; then
set +a set +a
fi fi
if [[ -x "$ENSURE_XMPP_SECRET_SCRIPT" ]]; then
XMPP_SECRET="$("$ENSURE_XMPP_SECRET_SCRIPT" "$STACK_ENV")"
export XMPP_SECRET
fi
STACK_ID="${GIA_STACK_ID:-${STACK_ID:-}}" STACK_ID="${GIA_STACK_ID:-${STACK_ID:-}}"
STACK_ID="$(echo "$STACK_ID" | tr -cs 'a-zA-Z0-9._-' '-' | sed 's/^-*//; s/-*$//')" STACK_ID="$(echo "$STACK_ID" | tr -cs 'a-zA-Z0-9._-' '-' | sed 's/^-*//; s/-*$//')"
@@ -29,21 +35,41 @@ PROSODY_CONFIG_FILE="${QUADLET_PROSODY_CONFIG_FILE:-$ROOT_DIR/utilities/prosody/
PROSODY_CERTS_DIR="${QUADLET_PROSODY_CERTS_DIR:-$ROOT_DIR/.podman/gia_prosody_certs}" PROSODY_CERTS_DIR="${QUADLET_PROSODY_CERTS_DIR:-$ROOT_DIR/.podman/gia_prosody_certs}"
PROSODY_DATA_DIR="${QUADLET_PROSODY_DATA_DIR:-$ROOT_DIR/.podman/gia_prosody_data}" PROSODY_DATA_DIR="${QUADLET_PROSODY_DATA_DIR:-$ROOT_DIR/.podman/gia_prosody_data}"
PROSODY_LOGS_DIR="${QUADLET_PROSODY_LOGS_DIR:-$ROOT_DIR/.podman/gia_prosody_logs}" PROSODY_LOGS_DIR="${QUADLET_PROSODY_LOGS_DIR:-$ROOT_DIR/.podman/gia_prosody_logs}"
PROSODY_IMAGE="${PROSODY_IMAGE:-docker.io/prosody/prosody-alpine:latest}"
mkdir -p "$PROSODY_CERTS_DIR" "$PROSODY_DATA_DIR" "$PROSODY_LOGS_DIR" mkdir -p "$PROSODY_CERTS_DIR" "$PROSODY_DATA_DIR" "$PROSODY_LOGS_DIR"
up() { up() {
local run_args=()
local pod_state=""
if podman pod exists "$POD_NAME"; then
pod_state="$(podman pod inspect "$POD_NAME" --format '{{.State}}' 2>/dev/null || true)"
if [[ "$pod_state" == "Running" ]]; then
run_args+=(--pod "$POD_NAME")
else
echo "Warning: pod '$POD_NAME' state is '$pod_state'; starting $PROSODY_CONTAINER standalone with explicit ports." >&2
run_args+=(-p 5222:5222 -p 5269:5269 -p 5280:5280 -p 8888:8888)
fi
else
echo "Warning: pod '$POD_NAME' not found; starting $PROSODY_CONTAINER standalone with explicit ports." >&2
run_args+=(-p 5222:5222 -p 5269:5269 -p 5280:5280 -p 8888:8888)
fi
podman run -d \ podman run -d \
--replace \ --replace \
--name "$PROSODY_CONTAINER" \ --name "$PROSODY_CONTAINER" \
--pod "$POD_NAME" \ "${run_args[@]}" \
--env-file "$STACK_ENV" \
-v "$PROSODY_CONFIG_FILE:/etc/prosody/prosody.cfg.lua:ro" \ -v "$PROSODY_CONFIG_FILE:/etc/prosody/prosody.cfg.lua:ro" \
-v "$PROSODY_CERTS_DIR:/etc/prosody/certs" \ -v "$PROSODY_CERTS_DIR:/etc/prosody/certs" \
-v "$PROSODY_DATA_DIR:/var/lib/prosody" \ -v "$PROSODY_DATA_DIR:/var/lib/prosody" \
-v "$PROSODY_LOGS_DIR:/var/log/prosody" \ -v "$PROSODY_LOGS_DIR:/var/log/prosody" \
-v "$ROOT_DIR:/code" \ -v "$ROOT_DIR:/code" \
docker.io/prosody/prosody:0.12 >/dev/null "$PROSODY_IMAGE" >/dev/null
echo "Started $PROSODY_CONTAINER in pod $POD_NAME" if [[ " ${run_args[*]} " == *" --pod "* ]]; then
echo "Started $PROSODY_CONTAINER in pod $POD_NAME"
else
echo "Started $PROSODY_CONTAINER standalone (not attached to pod $POD_NAME)"
fi
} }
down() { down() {

22
utilities/prosody/prosody.cfg.lua
View File

@@ -1,9 +1,19 @@
local env = os.getenv
local domain = env("DOMAIN") or "example.com"
local xmpp_component = env("XMPP_JID") or ("jews." .. domain)
local share_host = env("XMPP_SHARE_HOST") or ("share." .. domain)
local xmpp_secret = env("XMPP_SECRET") or ""
if xmpp_secret == "" then
error("XMPP_SECRET is required for Prosody component authentication")
end
sasl_mechanisms = { "PLAIN", "SCRAM-SHA-1", "SCRAM-SHA-256" } sasl_mechanisms = { "PLAIN", "SCRAM-SHA-1", "SCRAM-SHA-256" }
daemonize = false daemonize = false
pidfile = "/run/prosody/prosody.pid" pidfile = "/run/prosody/prosody.pid"
admins = { "mm@zm.is" } admins = { env("XMPP_ADMIN_JID") or ("admin@" .. domain) }
modules_enabled = { modules_enabled = {
"disco"; "disco";
@@ -59,16 +69,16 @@ certificates = "certs"
component_ports = { 8888 } component_ports = { 8888 }
component_interfaces = { "0.0.0.0" } component_interfaces = { "0.0.0.0" }
VirtualHost "zm.is" VirtualHost domain
authentication = "external_insecure" authentication = "external_insecure"
external_auth_command = "/code/utilities/prosody/auth_django.sh" external_auth_command = "/code/utilities/prosody/auth_django.sh"
certificate = "/etc/prosody/certs/cert.pem" certificate = "/etc/prosody/certs/cert.pem"
Component "jews.zm.is" Component xmpp_component
component_secret = "REepvw+QeX3ZzfmRSbBMKQhyiPd5bFowesnYuiiYbiYy2ZQVXvayxmsB" component_secret = xmpp_secret
Component "share.zm.is" "http_file_share" Component share_host "http_file_share"
http_ports = { 5280 } http_ports = { 5280 }
http_interfaces = { "0.0.0.0", "::" } http_interfaces = { "0.0.0.0", "::" }
http_external_url = "https://share.zm.is/" http_external_url = "https://" .. share_host .. "/"

114
utilities/prosody/renew_prosody_cert.sh
View File

@@ -3,8 +3,13 @@ set -euo pipefail
# Run as root from host. This script pipes certificate material through the # Run as root from host. This script pipes certificate material through the
# `code` user into the Prosody container via podman exec. # `code` user into the Prosody container via podman exec.
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
DOMAIN="${DOMAIN:-zm.is}" STACK_ENV="${STACK_ENV:-$ROOT_DIR/stack.env}"
if [[ -f "$STACK_ENV" ]]; then
set -a
. "$STACK_ENV"
set +a
fi
STACK_ID="${GIA_STACK_ID:-${STACK_ID:-}}" STACK_ID="${GIA_STACK_ID:-${STACK_ID:-}}"
STACK_ID="$(echo "$STACK_ID" | tr -cs 'a-zA-Z0-9._-' '-' | sed 's/^-*//; s/-*$//')" STACK_ID="$(echo "$STACK_ID" | tr -cs 'a-zA-Z0-9._-' '-' | sed 's/^-*//; s/-*$//')"
@@ -14,16 +19,101 @@ else
PROSODY_CONTAINER_DEFAULT="prosody_gia" PROSODY_CONTAINER_DEFAULT="prosody_gia"
fi fi
PROSODY_CONTAINER="${PROSODY_CONTAINER:-$PROSODY_CONTAINER_DEFAULT}" PROSODY_CONTAINER="${PROSODY_CONTAINER:-$PROSODY_CONTAINER_DEFAULT}"
MANAGE_SCRIPT="${MANAGE_SCRIPT:-$ROOT_DIR/utilities/prosody/manage_prosody_container.sh}"
ACME_BASE_DIR="${ACME_BASE_DIR:-/root/.acme.sh}"
CERT_NAME="${CERT_NAME:-${ACME_CERT_NAME:-}}"
FULLCHAIN_PATH="${FULLCHAIN_PATH:-/root/.acme.sh/${DOMAIN}/fullchain.cer}" FULLCHAIN_PATH="${FULLCHAIN_PATH:-}"
KEY_PATH="${KEY_PATH:-/root/.acme.sh/${DOMAIN}/${DOMAIN}.key}" KEY_PATH="${KEY_PATH:-}"
CERT_PATH_IN_CONTAINER="${CERT_PATH_IN_CONTAINER:-/etc/prosody/certs/cert.pem}" CERT_PATH_IN_CONTAINER="${CERT_PATH_IN_CONTAINER:-/etc/prosody/certs/cert.pem}"
CONTAINER_WAIT_SECONDS="${CONTAINER_WAIT_SECONDS:-15}"
resolve_cert_paths() {
local cert_dir=""
local expected_key=""
if [[ -n "$CERT_NAME" ]]; then
cert_dir="$ACME_BASE_DIR/$CERT_NAME"
if [[ -r "$cert_dir/fullchain.cer" ]]; then
expected_key="$cert_dir/${CERT_NAME}.key"
if [[ -r "$expected_key" ]]; then
FULLCHAIN_PATH="$cert_dir/fullchain.cer"
KEY_PATH="$expected_key"
return 0
fi
KEY_PATH="$(find "$cert_dir" -maxdepth 1 -type f -name '*.key' | head -n1 || true)"
if [[ -n "$KEY_PATH" && -r "$KEY_PATH" ]]; then
FULLCHAIN_PATH="$cert_dir/fullchain.cer"
return 0
fi
fi
echo "Requested CERT_NAME '$CERT_NAME' does not provide readable fullchain/key under $cert_dir" >&2
return 1
fi
cert_dir="$(find "$ACME_BASE_DIR" -mindepth 1 -maxdepth 1 -type d \
-exec test -r '{}/fullchain.cer' ';' -printf '%T@ %p\n' \
| sort -nr \
| awk 'NR==1 {print $2}' || true)"
if [[ -z "$cert_dir" ]]; then
echo "No readable ACME certificate directories with fullchain.cer found under $ACME_BASE_DIR" >&2
return 1
fi
FULLCHAIN_PATH="$cert_dir/fullchain.cer"
KEY_PATH="$(find "$cert_dir" -maxdepth 1 -type f -name '*.key' | head -n1 || true)"
if [[ -z "$KEY_PATH" || ! -r "$KEY_PATH" ]]; then
echo "No readable key file (*.key) found in $cert_dir" >&2
return 1
fi
return 0
}
code_podman() {
su -s /bin/sh code -c "podman $*"
}
container_exists() {
code_podman "container exists '$PROSODY_CONTAINER'"
}
container_is_running() {
[[ "$(code_podman "inspect '$PROSODY_CONTAINER' --format '{{.State.Running}}'" 2>/dev/null || true)" == "true" ]]
}
ensure_running_container() {
if ! container_exists; then
echo "Prosody container '$PROSODY_CONTAINER' not found for user 'code'; attempting startup..." >&2
su -s /bin/sh code -c "cd '$ROOT_DIR/utilities/prosody' && '$MANAGE_SCRIPT' up"
fi
if ! container_exists; then
echo "Failed to create/start Prosody container: $PROSODY_CONTAINER" >&2
exit 1
fi
if ! container_is_running; then
code_podman "start '$PROSODY_CONTAINER'" >/dev/null 2>&1 || true
fi
local i=0
while (( i < CONTAINER_WAIT_SECONDS )); do
if container_is_running; then
return 0
fi
sleep 1
i=$((i + 1))
done
echo "Prosody container exists but is not running: $PROSODY_CONTAINER" >&2
code_podman "inspect '$PROSODY_CONTAINER' --format 'status={{.State.Status}} exit={{.State.ExitCode}} error={{.State.Error}}'" >&2 || true
echo "Recent Prosody logs:" >&2
code_podman "logs --tail 120 '$PROSODY_CONTAINER'" >&2 || true
exit 1
}
if [[ "$(id -u)" -ne 0 ]]; then if [[ "$(id -u)" -ne 0 ]]; then
echo "This script must run as root." >&2 echo "This script must run as root." >&2
exit 1 exit 1
fi fi
if [[ -z "$FULLCHAIN_PATH" || -z "$KEY_PATH" ]]; then
resolve_cert_paths
fi
if [[ ! -r "$FULLCHAIN_PATH" ]]; then if [[ ! -r "$FULLCHAIN_PATH" ]]; then
echo "Missing or unreadable fullchain: $FULLCHAIN_PATH" >&2 echo "Missing or unreadable fullchain: $FULLCHAIN_PATH" >&2
exit 1 exit 1
@@ -34,10 +124,22 @@ if [[ ! -r "$KEY_PATH" ]]; then
exit 1 exit 1
fi fi
ensure_running_container
cat "$FULLCHAIN_PATH" "$KEY_PATH" \ cat "$FULLCHAIN_PATH" "$KEY_PATH" \
| sed '/^$/d' \ | sed '/^$/d' \
| su -s /bin/sh code -c "podman exec -i $PROSODY_CONTAINER sh -lc 'cat > $CERT_PATH_IN_CONTAINER'" | su -s /bin/sh code -c "podman exec --user 0 -i '$PROSODY_CONTAINER' sh -lc 'cat > \"$CERT_PATH_IN_CONTAINER\"'"
su -s /bin/sh code -c "podman exec $PROSODY_CONTAINER sh -lc 'chown prosody:prosody $CERT_PATH_IN_CONTAINER && chmod 0600 $CERT_PATH_IN_CONTAINER && prosodyctl reload'" su -s /bin/sh code -c "podman exec --user 0 '$PROSODY_CONTAINER' sh -lc '
set -e
chown prosody:prosody \"$CERT_PATH_IN_CONTAINER\"
chmod 0600 \"$CERT_PATH_IN_CONTAINER\"
if prosodyctl reload >/dev/null 2>&1; then
exit 0
fi
# In foreground/container mode prosodyctl may report \"Prosody is not running\"
# despite PID 1 being the active prosody process. HUP PID 1 as reload fallback.
kill -HUP 1
'"
echo "Prosody certificate updated and reloaded in container: $PROSODY_CONTAINER" echo "Prosody certificate updated and reloaded in container: $PROSODY_CONTAINER"

163
vendor/django-crud-mixins/.gitignore vendored Normal file
View File

@@ -0,0 +1,163 @@
# ---> Python
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
*_ref*

30
vendor/django-crud-mixins/.pre-commit-config.yaml vendored Normal file
View File

@@ -0,0 +1,30 @@
repos:
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: black
exclude: ^core/migrations
- repo: https://github.com/PyCQA/isort
rev: 5.11.5
hooks:
- id: isort
args: ["--profile", "black"]
- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
hooks:
- id: flake8
args: [--max-line-length=88]
exclude: ^core/migrations
- repo: https://github.com/rtts/djhtml
rev: v2.0.0
hooks:
- id: djhtml
args: [-t 2]
- id: djcss
exclude : ^core/static/css # slow
- id: djjs
exclude: ^core/static/js # slow
- repo: https://github.com/sirwart/ripsecrets.git
rev: v0.1.5
hooks:
- id: ripsecrets

9
vendor/django-crud-mixins/LICENSE vendored Normal file
View File

@@ -0,0 +1,9 @@
MIT License
Copyright (c) 2025 Mark Veidemanis
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

3
vendor/django-crud-mixins/MANIFEST.in vendored Normal file
View File

@@ -0,0 +1,3 @@
include README.md
recursive-include mixins/templates *
recursive-include mixins/static *

393
vendor/django-crud-mixins/README.md vendored Normal file
View File

@@ -0,0 +1,393 @@
# 🚀 django-crud-mixins
**Reusable Django CRUD mixins** for rapid development of single-page applications (SPA) and grid-based UIs.
Designed for **HTMX**, **Gridstack.js**, and **Django ORM** with built-in **access control** and **permissions**.
## 📌 Features
- **Django 4+ support** with modular mixins
- **HTMX-friendly partials**
- **Pre-configured Gridstack.js widget support**
- **RBAC enforcement with per-user filtering**
- **Reusable mixins for List, Create, Read, Update, Delete**
- **Automatic permission enforcement on querysets**
- **Seamless integration with Django Forms and QuerySets**
- **Prevents unauthorized data access**
- **Extra Buttons for dynamic action-based UI extensions**
- **Context-aware detail views with Pretty Print formatting**
## 📑 CRUD Mixins Overview
| Mixin | Description |
|----------------------|--------------------------------------------------|
| `ObjectList` | List view with pagination and permission-based filtering |
| `ObjectCreate` | Create new objects with form validation |
| `ObjectRead` | Read-only detail view with pre-fetching |
| `ObjectUpdate` | Edit existing objects with permission enforcement |
| `ObjectDelete` | Soft-delete or hard-delete objects securely |
| `RestrictedViewMixin` | Enforces user-based filtering on querysets |
| `RestrictedFormMixin` | Auto-filters form choices based on user permissions |
---
## ⚙️ Installation
Add to your `requirements.txt`:
```shell
git+https://git.zm.is/XF/django-crud-mixins
```
Or install via pip:
```shell
pip install git+https://git.zm.is/XF/django-crud-mixins
```
## 🔧 Usage
### 📂 Import the CRUD mixins
```python
from mixins.views import ObjectCreate, ObjectDelete, ObjectList, ObjectRead, ObjectUpdate
```
### 📝 Define CRUD Views
```python
class AccountList(LoginRequiredMixin, ObjectList):
list_template = "partials/account-list.html"
model = Account
page_title = "List of accounts"
list_url_name = "accounts"
list_url_args = ["type"]
submit_url_name = "account_create"
class AccountCreate(LoginRequiredMixin, ObjectCreate):
model = Account
form_class = AccountForm
submit_url_name = "account_create"
class AccountUpdate(LoginRequiredMixin, ObjectUpdate):
model = Account
form_class = AccountForm
submit_url_name = "account_update"
class AccountDelete(LoginRequiredMixin, ObjectDelete):
model = Account
```
### Add to `urls.py`
```python
path("accounts/<str:type>/", accounts.AccountList.as_view(), name="accounts"),
path(
"accounts/<str:type>/create/",
accounts.AccountCreate.as_view(),
name="account_create",
),
path(
"accounts/<str:type>/update/<str:pk>/",
accounts.AccountUpdate.as_view(),
name="account_update",
),
path(
"accounts/<str:type>/delete/<str:pk>/",
accounts.AccountDelete.as_view(),
name="account_delete",
),
```
## ⚙️ Configuration Options
### General Options
| Variable | Description |
|-----------------|----------------------------------|
| `list_template` | Template name for list view |
| `model` | Django model used for the view |
| `page_title` | Title of the page |
| `page_subtitle` | Subtitle of the page |
### List URLs
| Variable | Description |
|----------------|----------------------------------|
| `list_url_name` | URL name for listing objects |
| `list_url_args` | URL arguments for list endpoint |
### Submit URLs (Forms)
| Variable | Description |
|----------------|------------------------------|
| `submit_url_name` | URL for submitting forms |
| `submit_url_args` | URL args for submitting |
### ObjectList-Specific Options
| Variable | Description |
|----------------------|------------------------------|
| `delete_all_url_name` | URL for bulk deletion |
| `widget_options` | Gridstack widget config |
### ObjectCreate & ObjectUpdate
| Variable | Description |
|--------------|----------------------------------|
| `hide_cancel` | Hide cancel button in forms |
### ObjectUpdate-Only
| Variable | Description |
|--------------|-----------------------------------|
| `pk_required` | Enforce primary key requirement |
### 🔄 Extra Buttons
The mixins allow dynamic **extra buttons** to be added to views for custom actions.
Extra buttons are rendered inside the `<div class="buttons">` section of each view:
```html
{% for button in extra_buttons %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-{{ button.method }}="{{ button.url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
{% if button.confirm %}hx-confirm="Are you sure you wish to {{ button.action }}?"{% endif %}
class="button">
<span class="icon-text">
<span class="icon">
<i class="{{ button.icon }}"></i>
</span>
<span>{{ button.label }}</span>
</span>
</button>
{% endfor %}
```
#### Structure of Extra Buttons
Each extra button is a dictionary with:
| Key | Description |
|-----------|----------------------------------------------|
| `method` | HTMX method (`get`, `post`, `delete`, etc.) |
| `url` | Endpoint to be triggered |
| `action` | Action name (for confirmation) |
| `icon` | CSS class for icon (FontAwesome recommended) |
| `label` | Display text for the button |
| `confirm` | Optional confirmation prompt |
#### Example Usage in Views
```python
class CustomListView(ObjectList):
extra_buttons = [
{
"method": "post",
"url": reverse_lazy("custom-action"),
"icon": "fa-solid fa-bolt",
"label": "Trigger Action",
"confirm": True,
"action": "trigger this action"
}
]
```
### 🖼️ Pretty Print Context (pretty)
The **Pretty Print** (pretty) context is used in `ObjectRead` detail views to enhance object representation.
It is loaded via `{% load pretty %}` and ensures **formatted output of complex data**.
Implement it into your project's `templatetags` folder to use:
```python
import orjson
from django import template
register = template.Library()
@register.filter
def pretty(data):
return orjson.dumps(data, option=orjson.OPT_INDENT_2).decode("utf-8")
```
#### When to Use Pretty Print
- **Complex JSON Fields** (e.g., structured responses)
- **Nested Dictionaries** (e.g., API outputs)
- **Formatted Logs & Debugging**
## 🔥 Extending Functionality
### ✅ Enforce Query Restrictions (RBAC)
Add extra permission filters via `extra_permission_args`:
```python
class CustomListView(ObjectList):
def set_extra_args(self, user):
self.extra_permission_args = {"organization": user.organization}
```
This ensures users can only see objects from their organization.
### 📍 Hook Into Save Events
Modify or abort saves via `pre_save_mutate` and `post_save`:
```python
class SecureObjectCreate(ObjectCreate):
def pre_save_mutate(self, user, obj):
if not user.is_admin:
raise AbortSave("You do not have permission to save this object.")
def post_save(self, obj):
send_notification(f"Object {obj.id} was created.")
```
## 🔒 Secure Form QuerySets
```python
from mixins.forms import RestrictedFormMixin
from django.forms import ModelForm
class YourModelForm(RestrictedFormMixin, ModelForm):
class Meta:
model = YourModel
fields = ["name"]
```
This ensures users **only see objects they have access** to in form dropdowns.
## 🔗 Nested Objects & Relationships
### Overview
Some objects exist **within a parent object**. In this case, views and URLs must be structured to reflect this relationship. The example below demonstrates managing **Person Identifiers**, which are linked to a **Person**.
### How It Works
1. **Filtering:** The `IdentifierPermissionMixin` ensures only identifiers belonging to a specific `Person` (and owned by the user) are retrieved.
2. **View Permissions:** The `set_extra_args` method filters data by enforcing **user ownership** and **parent object constraints**.
3. **Validation:**
- `PersonIdentifierCreate` ensures an identifier cannot be added twice.
- `PersonIdentifierUpdate` restricts edits to only valid, existing identifiers.
- `PersonIdentifierDelete` removes identifiers that belong to the requesting user.
4. **URL Structure:** Each identifier action (list, create, update, delete) includes the **Person ID**, ensuring operations are always scoped to the correct entity.
### Example: Managing Person Identifiers
#### 📌 Views
| View | Description |
|------|------------|
| `PersonIdentifierList` | Displays all identifiers linked to a specific `Person`. |
| `PersonIdentifierCreate` | Adds a new identifier for a `Person`, preventing duplicates. |
| `PersonIdentifierUpdate` | Edits an existing identifier, enforcing access control. |
| `PersonIdentifierDelete` | Deletes an identifier, ensuring ownership validation. |
#### 📌 URL Routing
| Route | Description |
|-------|------------|
| `/person/<str:type>/identifiers/<str:person>/` | Lists identifiers for a `Person`. |
| `/person/<str:type>/identifiers/create/<str:person>` | Creates a new identifier under a `Person`. |
| `/person/<str:type>/identifiers/update/<str:person>/<str:pk>/` | Updates a specific identifier. |
| `/person/<str:type>/identifiers/delete/<str:person>/<str:pk>/` | Deletes an identifier for a `Person`. |
### View
```python
class IdentifierPermissionMixin:
def set_extra_args(self, user):
self.extra_permission_args = {
"person__user": user,
"person__pk": self.kwargs["person"],
}
class PersonIdentifierList(LoginRequiredMixin, IdentifierPermissionMixin, ObjectList):
list_template = "partials/identifier-list.html"
model = PersonIdentifier
page_title = "Person Identifiers"
list_url_name = "person_identifiers"
list_url_args = ["type", "person"]
submit_url_name = "person_identifier_create"
submit_url_args = ["type", "person"]
class PersonIdentifierCreate(LoginRequiredMixin, IdentifierPermissionMixin, ObjectCreate):
model = PersonIdentifier
form_class = PersonIdentifierForm
submit_url_name = "person_identifier_create"
submit_url_args = ["type", "person"]
def form_valid(self, form):
"""If the form is invalid, render the invalid form."""
try:
return super().form_valid(form)
except IntegrityError as e:
if "UNIQUE constraint failed" in str(e):
form.add_error("identifier", "Identifier rule already exists")
return self.form_invalid(form)
else:
raise e
def pre_save_mutate(self, user, obj):
try:
person = Person.objects.get(pk=self.kwargs["person"], user=user)
obj.person = person
except Person.DoesNotExist:
log.error(f"Person {self.kwargs['person']} does not exist")
raise AbortSave("person does not exist or you don't have access")
class PersonIdentifierUpdate(LoginRequiredMixin, IdentifierPermissionMixin, ObjectUpdate):
model = PersonIdentifier
form_class = PersonIdentifierForm
submit_url_name = "person_identifier_update"
submit_url_args = ["type", "pk", "person"]
class PersonIdentifierDelete(LoginRequiredMixin, IdentifierPermissionMixin, ObjectDelete):
model = PersonIdentifier
```
#### 📌 Button Example (Template)
To display a button linking to a **Person’s Identifiers**:
```html
<a href="{% url 'person_identifiers' type='page' person=item.id %}">
<button class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-eye"></i>
</span>
</span>
</button>
</a>
```
## 🚀 Example Templates
### 🔥 List Template (partials/account-list.html)
```html
<table class="table is-striped">
<thead>
<tr>
<th>Account</th>
<th>Email</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for account in object_list %}
<tr>
<td>{{ account.name }}</td>
<td>{{ account.email }}</td>
<td>
<a href="{% url 'account_update' pk=account.pk %}" class="button is-small">Edit</a>
<a href="{% url 'account_delete' pk=account.pk %}" class="button is-danger is-small">Delete</a>
</td>
</tr>
{% empty %}
<tr>
<td colspan="3">No accounts found.</td>
</tr>
{% endfor %}
</tbody>
</table>
```
## ⚠️ Legal Disclaimer
**django-crud-mixins is provided "as is", without warranty of any kind.**
Use responsibly and comply with local data privacy laws. The maintainers are **not responsible** for misuse.

0
vendor/django-crud-mixins/mixins/__init__.py vendored Normal file
View File

0
vendor/django-crud-mixins/mixins/__main__.py vendored Normal file
View File

100
vendor/django-crud-mixins/mixins/restrictions.py vendored Normal file
View File

@@ -0,0 +1,100 @@
from django.core.exceptions import FieldDoesNotExist, ImproperlyConfigured
from django.core.paginator import Paginator
from django.db.models import QuerySet
class RestrictedViewMixin:
"""
This mixin overrides two helpers in order to pass the user object to the filters.
get_queryset alters the objects returned for list views.
get_form_kwargs passes the request object to the form class. Remaining permissions
checks are in forms.py
"""
allow_empty = True
queryset = None
model = None
paginate_by = None
paginate_orphans = 0
context_object_name = None
paginator_class = Paginator
page_kwarg = "page"
ordering = None
def set_extra_args(self, user):
"""
This function is overriden to filter the objects by the requesting user.
"""
self.extra_permission_args = {}
def get_queryset(self, **kwargs):
"""
This function is overriden to filter the objects by the requesting user.
"""
self.set_extra_args(self.request.user)
if self.queryset is not None:
queryset = self.queryset
if isinstance(queryset, QuerySet):
# queryset = queryset.all()
queryset = queryset.filter(
user=self.request.user, **self.extra_permission_args
)
elif self.model is not None:
queryset = self.model._default_manager.filter(
user=self.request.user, **self.extra_permission_args
)
else:
raise ImproperlyConfigured(
"%(cls)s is missing a QuerySet. Define "
"%(cls)s.model, %(cls)s.queryset, or override "
"%(cls)s.get_queryset()." % {"cls": self.__class__.__name__}
)
if hasattr(self, "get_ordering"):
ordering = self.get_ordering()
if ordering:
if isinstance(ordering, str):
ordering = (ordering,)
queryset = queryset.order_by(*ordering)
return queryset
def get_form_kwargs(self):
"""Passes the request object to the form class.
This is necessary to only display members that belong to a given user"""
kwargs = super().get_form_kwargs()
kwargs["request"] = self.request
return kwargs
class RestrictedFormMixin:
"""
This mixin is used to restrict the queryset of a form to the current user.
The request object is passed from the view.
Fieldargs is used to pass additional arguments to the queryset filter.
"""
fieldargs = {}
# TODO: implement set_extra_args to check more permissions here
# for completeness, however as views open forms, the permissions
# are already checked there, so it may not be necessary.
def __init__(self, *args, **kwargs):
# self.fieldargs = {}
self.request = kwargs.pop("request")
super().__init__(*args, **kwargs)
for field in self.fields:
# Check it's not something like a CharField which has no queryset
if not hasattr(self.fields[field], "queryset"):
continue
model = self.fields[field].queryset.model
# Check if the model has a user field
try:
model._meta.get_field("user")
# Add the user to the queryset filters
self.fields[field].queryset = model.objects.filter(
user=self.request.user, **self.fieldargs.get(field, {})
)
except FieldDoesNotExist:
pass

1
vendor/django-crud-mixins/mixins/templates/mixins/partials/close-modal.html vendored Normal file
View File

@@ -0,0 +1 @@
<button class="modal-close is-large" aria-label="close"></button>

3
vendor/django-crud-mixins/mixins/templates/mixins/partials/close-widget.html vendored Normal file
View File

@@ -0,0 +1,3 @@
<i
class="fa-solid fa-xmark has-text-grey-light float-right"
onclick='grid.removeWidget("widget-{{ unique }}");grid.compact();'></i>

3
vendor/django-crud-mixins/mixins/templates/mixins/partials/close-window.html vendored Normal file
View File

@@ -0,0 +1,3 @@
<i
class="fa-solid fa-xmark has-text-grey-light float-right"
data-script="on click remove the closest <nav/>"></i>

77
vendor/django-crud-mixins/mixins/templates/mixins/partials/generic-detail.html vendored Normal file
View File

@@ -0,0 +1,77 @@
{% load pretty %}
{% load cache %}
{% cache 600 generic_detail request.user.id object live %}
{% include 'mixins/partials/notify.html' %}
{% if live is not None %}
<h1 class="title">Live {{ context_object_name_singular }} info</h1>
<table class="table is-fullwidth is-hoverable">
<thead>
<th>attribute</th>
<th>value</th>
</thead>
<tbody>
{% block live_tbody %}
{% for key, item in live.items %}
{% if key in pretty %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
<pre>{{ item|pretty }}</pre>
{% endif %}
</td>
</tr>
{% else %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
{{ item }}
{% endif %}
</td>
</tr>
{% endif %}
{% endfor %}
{% endblock %}
</tbody>
</table>
{% endif %}
{% if object is not None %}
<h1 class="title">{{ title_singular }} info</h1>
<table class="table is-fullwidth is-hoverable">
<thead>
<th>attribute</th>
<th>value</th>
</thead>
<tbody>
{% block tbody %}
{% for key, item in object.items %}
{% if key in pretty %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
<pre>{{ item|pretty }}</pre>
{% endif %}
</td>
</tr>
{% else %}
<tr>
<th>{{ key }}</th>
<td>
{% if item is not None %}
{{ item }}
{% endif %}
</td>
</tr>
{% endif %}
{% endfor %}
{% endblock %}
</tbody>
</table>
{% endif %}
{% endcache %}

7
vendor/django-crud-mixins/mixins/templates/mixins/partials/notify.html vendored Normal file
View File

@@ -0,0 +1,7 @@
<div id="notification">
{% if message is not None %}
<div class="notification is-{{ class }}" hx-ext="remove-me" remove-me="3s">
{{ message }}
</div>
{% endif %}
</div>

26
vendor/django-crud-mixins/mixins/templates/mixins/window-content/object-form.html vendored Normal file
View File

@@ -0,0 +1,26 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
{% load crispy_forms_tags %}
{% load crispy_forms_bulma_field %}
<form
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-post="{{ submit_url }}"
hx-target="#modals-here"
hx-swap="innerHTML">
{% csrf_token %}
{{ form|crispy }}
{% if hide_cancel is not True %}
<button
type="button"
class="button is-light modal-close-button">
Cancel
</button>
{% endif %}
<button type="submit" class="button modal-close-button">Submit</button>
</form>

61
vendor/django-crud-mixins/mixins/templates/mixins/window-content/object.html vendored Normal file
View File

@@ -0,0 +1,61 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
<div class="buttons">
{% if submit_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-get="{{ submit_url }}"
hx-trigger="click"
hx-target="#modals-here"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-plus"></i>
</span>
<span>{{ title_singular }}</span>
</span>
</button>
{% endif %}
{% if delete_all_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-delete="{{ delete_all_url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
hx-confirm="Are you sure you wish to delete all {{ context_object_name }}?"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-xmark"></i>
</span>
<span>Delete all {{ context_object_name }} </span>
</span>
</button>
{% endif %}
{% for button in extra_buttons %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-{{ button.method }}="{{ button.url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
{% if button.confirm %}hx-confirm="Are you sure you wish to {{ button.action }}?"{% endif %}
class="button">
<span class="icon-text">
<span class="icon">
<i class="{{ button.icon }}"></i>
</span>
<span>{{ button.label }}</span>
</span>
</button>
{% endfor %}
</div>
{% include detail_template %}

61
vendor/django-crud-mixins/mixins/templates/mixins/window-content/objects.html vendored Normal file
View File

@@ -0,0 +1,61 @@
{% include 'mixins/partials/notify.html' %}
{% if page_title is not None %}
<h1 class="title is-4">{{ page_title }}</h1>
{% endif %}
{% if page_subtitle is not None %}
<h1 class="subtitle">{{ page_subtitle }}</h1>
{% endif %}
<div class="buttons">
{% if submit_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-get="{{ submit_url }}"
hx-trigger="click"
hx-target="#modals-here"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-plus"></i>
</span>
<span>{{ title_singular }}</span>
</span>
</button>
{% endif %}
{% if delete_all_url is not None %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-delete="{{ delete_all_url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
hx-confirm="Are you sure you wish to delete all {{ context_object_name }}?"
class="button">
<span class="icon-text">
<span class="icon">
<i class="fa-solid fa-xmark"></i>
</span>
<span>Delete all {{ context_object_name }} </span>
</span>
</button>
{% endif %}
{% for button in extra_buttons %}
<button
hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'
hx-{{ button.method }}="{{ button.url }}"
hx-trigger="click"
hx-target="#modals-here"
hx-swap="innerHTML"
{% if button.confirm %}hx-confirm="Are you sure you wish to {{ button.action }}?"{% endif %}
class="button">
<span class="icon-text">
<span class="icon">
<i class="{{ button.icon }}"></i>
</span>
<span>{{ button.label }}</span>
</span>
</button>
{% endfor %}
</div>
{% include list_template %}

20
vendor/django-crud-mixins/mixins/templates/mixins/wm/modal.html vendored Normal file
View File

@@ -0,0 +1,20 @@
{% load static %}
<script src="{% static 'modal.js' %}"></script>
{% block scripts %}
{% endblock %}
{% block styles %}
{% endblock %}
<div id="modal" class="modal is-active is-clipped">
<div class="modal-background"></div>
<div class="modal-content">
<div class="box">
{% block modal_content %}
{% include window_content %}
{% endblock %}
{% include 'mixins/partials/close-modal.html' %}
</div>
</div>
</div>

6
vendor/django-crud-mixins/mixins/templates/mixins/wm/page.html vendored Normal file
View File

@@ -0,0 +1,6 @@
{% extends "base.html" %}
{% block content %}
{% include window_content %}
{% endblock %}

17
vendor/django-crud-mixins/mixins/templates/mixins/wm/panel.html vendored Normal file
View File

@@ -0,0 +1,17 @@
<nav class="panel">
<p class="panel-heading" style="padding: .2em; line-height: .5em;">
<i class="fa-solid fa-arrows-up-down-left-right has-text-grey-light"></i>
{% block close_button %}
{% include 'mixins/partials/close-window.html' %}
{% endblock %}
{% block heading %}
{% endblock %}
</p>
<article class="panel-block is-active">
<div class="control">
{% block panel_content %}
{% endblock %}
</div>
</article>
</nav>

37
vendor/django-crud-mixins/mixins/templates/mixins/wm/widget.html vendored Normal file
View File

@@ -0,0 +1,37 @@
<div id="widget">
<div id="widget-{{ unique }}" class="grid-stack-item" {% block widget_options %}{% if widget_options is None %}gs-w="6" gs-h="1" gs-y="20" gs-x="0"{% else %}{% autoescape off %}{{ widget_options }}{% endautoescape %}{% endif %}{% endblock %}>
<div class="grid-stack-item-content">
<nav class="panel">
<p class="panel-heading" style="padding: .2em; line-height: .5em;">
<i class="fa-solid fa-arrows-up-down-left-right has-text-grey-light"></i>
{% block close_button %}
{% include 'mixins/partials/close-widget.html' %}
{% endblock %}
<i
class="fa-solid fa-arrows-minimize has-text-grey-light float-right"
onclick='grid.compact();'></i>
{% block heading %}
{{ title }}
{% endblock %}
</p>
<article class="panel-block is-active">
<div class="control">
{% block panel_content %}
{% include window_content %}
{% endblock %}
</div>
</article>
</nav>
</div>
</div>
</div>
<script>
{% block custom_script %}
{% endblock %}
var widget_event = new Event('load-widget');
document.dispatchEvent(widget_event);
</script>
{% block custom_end %}
{% endblock %}

10
vendor/django-crud-mixins/mixins/templates/mixins/wm/window.html vendored Normal file
View File

@@ -0,0 +1,10 @@
<magnet-block attract-distance="10" align-to="outer|center" class="floating-window">
{% extends 'mixins/wm/panel.html' %}
{% block heading %}
{{ title }}
{% endblock %}
{% block panel_content %}
{% include window_content %}
{% endblock %}
</magnet-block>

466
vendor/django-crud-mixins/mixins/views.py vendored Normal file
View File

@@ -0,0 +1,466 @@
import uuid
from django.http import Http404, HttpResponse, HttpResponseBadRequest
from django.urls import reverse
from django.views.generic.detail import DetailView
from django.views.generic.edit import CreateView, DeleteView, UpdateView
from django.views.generic.list import ListView
from rest_framework.parsers import FormParser
from mixins.restrictions import RestrictedViewMixin
class AbortSave(Exception):
pass
class ObjectNameMixin(object):
def __init__(self, *args, **kwargs):
if self.model is None:
self.title = self.context_object_name.title()
self.title_singular = self.context_object_name_singular.title()
else:
self.title_singular = self.model._meta.verbose_name.title() # Hook
self.context_object_name_singular = self.title_singular.lower() # hook
self.title = self.model._meta.verbose_name_plural.title() # Hooks
self.context_object_name = self.title.lower() # hooks
self.context_object_name = self.context_object_name.replace(" ", "")
self.context_object_name_singular = (
self.context_object_name_singular.replace(" ", "")
)
super().__init__(*args, **kwargs)
class ObjectList(RestrictedViewMixin, ObjectNameMixin, ListView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/objects.html"
list_template = None
page_title = None
page_subtitle = None
list_url_name = None
# WARNING: TAKEN FROM locals()
list_url_args = ["type"]
submit_url_name = None
submit_url_args = ["type"]
delete_all_url_name = None
widget_options = None
extra_buttons = None
def queryset_mutate(self, queryset):
pass
# copied from BaseListView
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.request = request
self.object_list = self.get_queryset(**kwargs)
if isinstance(self.object_list, HttpResponse):
return self.object_list
if isinstance(self.object_list, HttpResponseBadRequest):
return self.object_list
self.queryset_mutate(self.object_list)
allow_empty = self.get_allow_empty()
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
list_url_args = {}
for arg in self.list_url_args:
if arg in locals():
list_url_args[arg] = locals()[arg]
elif arg in kwargs:
list_url_args[arg] = kwargs[arg]
orig_type = type
if type == "page":
type = "modal"
if not allow_empty:
# When pagination is enabled and object_list is a queryset,
# it's better to do a cheap query than to load the unpaginated
# queryset in memory.
if self.get_paginate_by(self.object_list) is not None and hasattr(
self.object_list, "exists"
):
is_empty = not self.object_list.exists()
else:
is_empty = not self.object_list
if is_empty:
raise Http404("Empty list")
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
context = self.get_context_data()
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["list_template"] = self.list_template
context["page_title"] = self.page_title
context["page_subtitle"] = self.page_subtitle
context["type"] = type
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
if self.submit_url_name is not None:
context["submit_url"] = reverse(
self.submit_url_name, kwargs=submit_url_args
)
if self.list_url_name is not None:
context["list_url"] = reverse(self.list_url_name, kwargs=list_url_args)
if self.delete_all_url_name:
context["delete_all_url"] = reverse(self.delete_all_url_name)
if self.widget_options:
context["widget_options"] = self.widget_options
if self.extra_buttons is not None:
context["extra_buttons"] = self.extra_buttons
# Return partials for HTMX
if self.request.htmx:
if request.headers["HX-Target"] == self.context_object_name + "-table":
self.template_name = self.list_template
elif orig_type == "page":
self.template_name = self.list_template
else:
context["window_content"] = self.list_template
return self.render_to_response(context)
def post(self, request, *args, **kwargs):
self.request = request
return super().get(request, *args, **kwargs)
class ObjectCreate(RestrictedViewMixin, ObjectNameMixin, CreateView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object-form.html"
parser_classes = [FormParser]
page_title = None
page_subtitle = None
model = None
submit_url_name = None
submit_url_args = ["type"]
request = None
# Whether to hide the cancel button in the form
hide_cancel = False
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.title = "Create " + self.context_object_name_singular
def post_save(self, obj):
pass
def pre_save_mutate(self, user, obj):
pass
def form_valid(self, form):
obj = form.save(commit=False)
if self.request is None:
raise Exception("Request is None")
obj.user = self.request.user
try:
self.pre_save_mutate(self.request.user, obj)
except AbortSave as e:
context = {"message": f"Failed to save: {e}", "class": "danger"}
return self.render_to_response(context)
obj.save()
form.save_m2m()
self.post_save(obj)
context = {"message": "Object created", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def form_invalid(self, form):
"""If the form is invalid, render the invalid form."""
return self.get(self.request, **self.kwargs, form=form)
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
self.request = request
self.kwargs = kwargs
if type == "widget":
self.hide_cancel = True
if type == "page":
type = "modal"
self.object = None
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
submit_url = reverse(self.submit_url_name, kwargs=submit_url_args)
context = self.get_context_data()
form = kwargs.get("form", None)
if form:
context["form"] = form
context["unique"] = unique
context["window_content"] = self.window_content
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
context["submit_url"] = submit_url
context["type"] = type
context["hide_cancel"] = self.hide_cancel
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
response = self.render_to_response(context)
# response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def post(self, request, *args, **kwargs):
self.request = request
self.template_name = "mixins/partials/notify.html"
return super().post(request, *args, **kwargs)
class ObjectRead(RestrictedViewMixin, ObjectNameMixin, DetailView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object.html"
detail_template = "mixins/partials/generic-detail.html"
page_title = None
page_subtitle = None
model = None
# submit_url_name = None
detail_url_name = None
# WARNING: TAKEN FROM locals()
detail_url_args = ["type"]
request = None
extra_buttons = None
def get(self, request, *args, **kwargs):
type = kwargs.get("type", None)
if not type:
return HttpResponseBadRequest("No type specified")
if type not in self.allowed_types:
return HttpResponseBadRequest()
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
detail_url_args = {}
for arg in self.detail_url_args:
if arg in locals():
detail_url_args[arg] = locals()[arg]
elif arg in kwargs:
detail_url_args[arg] = kwargs[arg]
self.request = request
self.kwargs = kwargs
self.object = self.get_object(**kwargs)
if isinstance(self.object, HttpResponse):
return self.object
orig_type = type
if type == "page":
type = "modal"
context = self.get_context_data()
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["detail_template"] = self.detail_template
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
context["type"] = type
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
if self.detail_url_name is not None:
context["detail_url"] = reverse(
self.detail_url_name, kwargs=detail_url_args
)
if self.extra_buttons is not None:
context["extra_buttons"] = self.extra_buttons
# Return partials for HTMX
if self.request.htmx:
if request.headers["HX-Target"] == self.context_object_name + "-info":
self.template_name = self.detail_template
elif orig_type == "page":
self.template_name = self.detail_template
else:
context["window_content"] = self.detail_template
return self.render_to_response(context)
def post(self, request, *args, **kwargs):
self.request = request
return super().get(request, *args, **kwargs)
class ObjectUpdate(RestrictedViewMixin, ObjectNameMixin, UpdateView):
allowed_types = ["modal", "widget", "window", "page"]
window_content = "mixins/window-content/object-form.html"
parser_classes = [FormParser]
page_title = None
page_subtitle = None
model = None
submit_url_name = None
submit_url_args = ["type", "pk"]
request = None
# Whether pk is required in the get request
pk_required = True
# Whether to hide the cancel button in the form
hide_cancel = False
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.title = "Update " + self.context_object_name_singular
def post_save(self, obj):
pass
def form_valid(self, form):
obj = form.save(commit=False)
if self.request is None:
raise Exception("Request is None")
obj.save()
form.save_m2m()
self.post_save(obj)
context = {"message": "Object updated", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def form_invalid(self, form):
"""If the form is invalid, render the invalid form."""
return self.get(self.request, **self.kwargs, form=form)
def get(self, request, *args, **kwargs):
self.request = request
type = kwargs.get("type", None)
pk = kwargs.get("pk", None)
if not type:
return HttpResponseBadRequest("No type specified")
if not pk:
if self.pk_required:
return HttpResponseBadRequest("No pk specified")
if type not in self.allowed_types:
return HttpResponseBadRequest("Invalid type specified")
self.template_name = f"mixins/wm/{type}.html"
unique = str(uuid.uuid4())[:8]
if type == "widget":
self.hide_cancel = True
if type == "page":
type = "modal"
self.object = self.get_object()
submit_url_args = {}
for arg in self.submit_url_args:
if arg in locals():
submit_url_args[arg] = locals()[arg]
elif arg in kwargs:
submit_url_args[arg] = kwargs[arg]
submit_url = reverse(self.submit_url_name, kwargs=submit_url_args)
context = self.get_context_data()
form = kwargs.get("form", None)
if form:
context["form"] = form
context["title"] = self.title + f" ({type})"
context["title_singular"] = self.title_singular
context["unique"] = unique
context["window_content"] = self.window_content
context["context_object_name"] = self.context_object_name
context["context_object_name_singular"] = self.context_object_name_singular
context["submit_url"] = submit_url
context["type"] = type
context["hide_cancel"] = self.hide_cancel
if self.page_title:
context["page_title"] = self.page_title
if self.page_subtitle:
context["page_subtitle"] = self.page_subtitle
response = self.render_to_response(context)
# response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
def post(self, request, *args, **kwargs):
self.request = request
self.template_name = "mixins/partials/notify.html"
return super().post(request, *args, **kwargs)
class ObjectDelete(RestrictedViewMixin, ObjectNameMixin, DeleteView):
model = None
template_name = "mixins/partials/notify.html"
# Overriden to prevent success URL from being used
def delete(self, request, *args, **kwargs):
"""
Call the delete() method on the fetched object and then redirect to the
success URL.
"""
self.object = self.get_object()
# success_url = self.get_success_url()
self.object.delete()
context = {"message": "Object deleted", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response
# This will be used in newer Django versions, until then we get a warning
def form_valid(self, form):
"""
Call the delete() method on the fetched object.
"""
self.object = self.get_object()
self.object.delete()
context = {"message": "Object deleted", "class": "success"}
response = self.render_to_response(context)
response["HX-Trigger"] = f"{self.context_object_name_singular}Event"
return response

5
vendor/django-crud-mixins/pyproject.toml vendored Normal file
View File

@@ -0,0 +1,5 @@
# pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

28
vendor/django-crud-mixins/setup.cfg vendored Normal file
View File

@@ -0,0 +1,28 @@
[metadata]
name = django-crud-mixins
version = 1.0.3
author = Mark Veidemanis
author_email = m@zm.is
url = https://git.zm.is/XF/django-crud-mixins
description = CRUD mixins for Django class-based views.
long_description = file: README.md
long_description_content_type = text/markdown
keywords = django, mixins, helpers, crud
license = MIT
classifiers =
License :: OSI Approved :: BSD License
Programming Language :: Python :: 3
Framework :: Django :: 4.1
Intended Audience :: Developers
[options]
packages = find:
zip_safe = True
include_package_data = True
install_requires =
django
django-rest-framework
[options.package_data]
mixins = templates/mixins/*, README.md
* = README.md