Multi-source OSINT data collection and parallel processing tool. Indexes 4chan, Discord and IRC, reorganizes the data into a common format, annotates language, sentiment and tokens in multiple threads, and outputs the results to Elasticsearch.
Go to file
Mark Veidemanis 5c3b338017 Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
docker Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
legacy Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
schemas Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
sources Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
.gitignore Add config directories to gitignore 2022-09-08 09:45:18 +01:00
.pre-commit-config.yaml Reinstate Redis cache 2022-09-04 21:38:53 +01:00
db.py Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
docker-compose.yml Implement ingesting to Redis from Threshold 2022-09-07 07:20:30 +01:00
monolith.py Implement threshold writing to Redis and manticore ingesting from Redis 2022-09-07 07:20:30 +01:00
requirements.txt Add aioredis 2022-09-08 09:44:27 +01:00
util.py Run processing in thread 2022-09-04 21:29:00 +01:00