Skip to main content

Audit Logs

Track all activity and changes within your organization for compliance and security.

Overview​

Audit logs provide a complete record of:

  • Query executions
  • Settings modifications
  • Team member actions
  • API key changes
  • Data exports

Feature Status​

🚧 Coming Soon

This feature is planned for a future release. The infrastructure is in place, but the UI and reporting features are still in development.

Planned Capabilities​

Activity Tracking​

All actions will be logged:

Query Activity

  • Who executed queries
  • When queries ran
  • Which providers used
  • Success/failure status
  • Cost per execution

Settings Changes

  • API key additions/deletions
  • Budget modifications
  • Organization profile updates
  • Member permission changes

Access Events

  • User logins
  • Failed login attempts
  • Password changes
  • API access attempts

Audit Log Viewer​

Dedicated audit log interface with:

Filtering

  • By user/team member
  • By action type
  • By date range
  • By resource (e.g., specific API key)

Search

  • Full-text search across logs
  • Filter by specific events
  • Complex query builder

Export

  • CSV export for compliance
  • JSON export for analysis
  • PDF reports for stakeholders

Retention & Compliance​

Data Retention

  • Configurable retention period
  • Minimum 90 days
  • Option for unlimited retention
  • Automatic archival

Compliance Features

  • SOC 2 audit trail
  • GDPR compliance (right to access)
  • HIPAA audit logging (if applicable)
  • Tamper-proof logs

Current Workarounds​

While the feature is in development, you can track activity manually:

Database Queries​

Query Supabase directly for activity:

-- Recent query executions
SELECT
  u.email as user_email,
  q.query_text,
  q.provider,
  q.created_at,
  r.total_cost
FROM llm_queries q
LEFT JOIN llm_responses r ON r.query_id = q.id
LEFT JOIN auth.users u ON u.id = q.user_id
ORDER BY q.created_at DESC
LIMIT 100;
-- API key changes (if tracked)
SELECT
  action,
  provider,
  created_at,
  created_by
FROM audit_logs
WHERE resource_type = 'api_key'
ORDER BY created_at DESC;

Provider Dashboards​

Check LLM provider dashboards for:

  • API key usage
  • Request counts
  • Cost tracking
  • Error logs

Anthropic: console.anthropic.com → Usage OpenAI: platform.openai.com → Usage Google: aistudio.google.com → Activity Perplexity: docs.perplexity.ai → Usage

Manual Logging​

For critical actions, maintain a manual log:

DateUserActionDetails
2025-01-15admin@company.comAdded API keyOpenAI GPT-4
2025-01-14team@company.comExecuted 50 queriesClaude Sonnet
2025-01-12admin@company.comUpdated budget$500/mo

Use Cases​

Security Auditing​

Goal: Detect unauthorized access or suspicious activity

Current Approach:

  1. Review Supabase auth logs
  2. Check query history for unusual patterns
  3. Monitor provider dashboards for unexpected usage
  4. Set up email alerts for budget overages

Compliance Reporting​

Goal: Demonstrate proper access controls and data handling

Current Approach:

  1. Export query history from Supabase
  2. Document team member access levels
  3. Screenshot API key management
  4. Maintain manual change log for critical actions

Cost Attribution​

Goal: Track which users/projects are incurring costs

Current Approach:

-- Cost by user
SELECT
  u.email,
  SUM(r.total_cost) as total_cost,
  COUNT(*) as query_count
FROM llm_responses r
JOIN llm_queries q ON q.id = r.query_id
LEFT JOIN auth.users u ON u.id = q.user_id
GROUP BY u.email
ORDER BY total_cost DESC;

Future Development​

Full audit logging will include:

Event Types​

Comprehensive event tracking:

Authentication

  • ✅ User login
  • ✅ User logout
  • ✅ Password reset
  • ✅ MFA enabled/disabled
  • ✅ API token created

Authorization

  • ✅ Permission granted
  • ✅ Permission revoked
  • ✅ Role changed
  • ✅ Access denied

Data Access

  • ✅ Query executed
  • ✅ Report generated
  • ✅ Data exported
  • ✅ Settings viewed

Configuration

  • ✅ API key added
  • ✅ API key deleted
  • ✅ Budget changed
  • ✅ Integration configured

Advanced Features​

Real-time Alerts

  • Notify on suspicious activity
  • Alert on policy violations
  • Budget threshold warnings
  • Failed access attempts

Anomaly Detection

  • Unusual query volumes
  • Unexpected providers used
  • Off-hours activity
  • Geographic anomalies

Compliance Reports

  • Pre-built compliance templates
  • Scheduled report delivery
  • Automated attestation
  • Evidence collection

Security & Privacy​

Data Protection​

Audit logs will be:

  • Encrypted at rest: AES-256 encryption
  • Immutable: Cannot be modified after creation
  • Tamper-proof: Cryptographic signatures
  • Backed up: Regular automated backups

Access Control​

Who can view audit logs:

  • Admins: Full access to all logs
  • Members: Can view own activity only
  • Viewers: No access to audit logs
  • Auditors: Read-only access (future role)

Privacy Considerations​

Audit logs include:

  • ✅ User identifiers (email, ID)
  • ✅ Action timestamps
  • ✅ Resource affected
  • ✅ IP addresses (optional)
  • ❌ Query content (unless explicitly enabled)
  • ❌ Response content (privacy)

Best Practices​

Review Regularly​

Schedule periodic audit reviews:

  • Weekly: Check recent high-cost actions
  • Monthly: Full activity review
  • Quarterly: Compliance audit
  • Annually: Comprehensive security review

Set Up Alerts​

Configure notifications for:

  • Failed login attempts (3+ in 1 hour)
  • API key changes
  • Budget exceeded
  • Large batch executions (100+ queries)
  • Off-hours access

Document Policies​

Maintain written policies for:

  • Who can access audit logs
  • How long logs are retained
  • What triggers an investigation
  • Incident response procedures

Export for Compliance​

Regular exports for compliance:

  • Monthly cost reports
  • Quarterly access reviews
  • Annual compliance reports
  • Incident investigations

Contributing​

Want to help build audit logging?

  1. Share your compliance requirements
  2. Contribute sample queries
  3. Review privacy/security approach
  4. Test beta features when available

Next Steps​

While the feature is in development:

  1. Set up manual logging for critical actions
  2. Configure provider dashboard alerts
  3. Document your compliance requirements
  4. Query Supabase directly for activity reports