diff --git a/claude-plan-phase-1.md b/claude-plan-phase-1.md new file mode 100644 index 0000000..56ad52d --- /dev/null +++ b/claude-plan-phase-1.md @@ -0,0 +1,392 @@ +# Progressive Phase 1 Implementation Plan: Incremental Infrastructure Setup + +## Overview +Instead of setting up all infrastructure at once, we'll progressively build Phase 1 in three distinct sub-phases, allowing for thorough testing and validation at each layer before adding complexity. + +## Phase 1A: Backend + Database Foundation (5-7 days) + +### Core Infrastructure Setup +**Dependencies: None** + +**1A.1 Go Project Structure & Basic Services** +- Create complete Go module structure for all microservices +- Implement basic Fiber apps with health endpoints for each service +- Set up shared packages structure (auth, database, observability, security) +- Create basic service discovery and configuration management +- Add graceful shutdown and signal handling + +**1A.2 Database Layer with Ent** +- Define complete Ent schemas (Question, GameSession, QuestionAttempt) +- Set up dual database configuration (SQLite for dev, PostgreSQL for testing) +- Create migration scripts and database initialization +- Implement basic CRUD operations for each entity +- Add database connection pooling and health checks + +**1A.3 Mock Authentication System** +- Create simple JWT-based mock authentication (no external calls) +- Implement basic role-based access control (player/admin) +- Add middleware for token validation using local secret +- Create user context propagation through services +- Add basic security headers and CORS + +**1A.4 Basic Docker Environment** +- Create `docker-compose.basic.yml` with only PostgreSQL +- Add database initialization and seeding containers +- Create health check scripts for database connectivity +- Set up volume management for persistent data + +**1A.5 Integration Testing Framework** +- Set up testcontainers for PostgreSQL integration tests +- Create test utilities for database seeding and cleanup +- Implement service-level integration tests +- Add API endpoint testing with mock authentication +- Create test data factories and fixtures + +**Success Criteria for 1A:** +- All services start and respond to health checks +- Database migrations run successfully +- Mock authentication validates tokens correctly +- Integration tests pass with PostgreSQL container +- Services can perform basic CRUD operations +- Docker environment runs consistently + +--- + +## Phase 1B: Add Redis Layer (2-3 days) + +### Redis Integration +**Dependencies: Phase 1A complete** + +**1B.1 Redis Configuration & Connection** +- Add Redis client to shared package with connection pooling +- Create Redis health checks and monitoring +- Implement connection retry logic and circuit breaker +- Add Redis configuration management (cluster vs single instance) + +**1B.2 Session Management with Redis** +- Migrate session storage from in-memory to Redis +- Implement session middleware with Redis backend +- Add session invalidation and cleanup routines +- Create session security (encryption, tampering protection) + +**1B.3 Caching Layer Implementation** +- Create cache abstraction layer for future use +- Add caching for frequently accessed data (questions, leaderboards) +- Implement cache invalidation strategies +- Add cache performance metrics + +**1B.4 Update Development Environment** +- Extend `docker-compose.basic.yml` to include Redis +- Create `docker-compose.redis.yml` configuration +- Update health check scripts to verify Redis connectivity +- Add Redis monitoring and debugging tools + +**1B.5 Redis Integration Testing** +- Extend integration tests to include Redis functionality +- Test session persistence across service restarts +- Validate cache behavior and invalidation +- Test Redis failover scenarios +- Add performance testing for cache operations + +**Success Criteria for 1B:** +- Redis connects reliably with proper failover +- Sessions persist correctly in Redis +- Cache operations perform as expected +- Integration tests pass with Redis + PostgreSQL +- Services handle Redis connectivity issues gracefully + +--- + +## Phase 1C: Add Zitadel Authentication (3-4 days) + +### External Authentication Integration +**Dependencies: Phase 1B complete** + +**1C.1 Zitadel Repository Implementation** +- Implement full ZitadelRepository interface with HTTP client +- Add JWT token validation with Zitadel public keys +- Create token refresh and user management functionality +- Implement circuit breaker for external API calls +- Add proper error handling and retry logic + +**1C.2 Replace Mock Authentication** +- Migrate from mock JWT to real Zitadel integration +- Update middleware to validate tokens against Zitadel +- Implement proper role and permission mapping +- Add MFA validation for admin endpoints +- Create user context enrichment from Zitadel claims + +**1C.3 Complete Docker Environment** +- Add Zitadel container to Docker Compose setup +- Create `docker-compose.yml` with full infrastructure stack +- Configure Zitadel with proper OAuth applications +- Set up development user accounts and roles +- Add monitoring for all services + +**1C.4 Observability Foundation** +- Add Prometheus metrics collection to all services +- Create basic Grafana dashboards +- Implement distributed tracing preparation +- Add structured logging across services +- Create alerting for critical failures + +**1C.5 End-to-End Integration Testing** +- Create full authentication flow tests +- Test token validation and refresh cycles +- Validate role-based access control +- Test multi-service communication with real auth +- Add performance testing under load + +**Success Criteria for 1C:** +- Zitadel authentication works end-to-end +- All services integrate properly with real JWT validation +- Role-based access control functions correctly +- Integration tests pass with full infrastructure stack +- System handles authentication failures gracefully +- Basic observability provides useful insights + +--- + +## Progressive Testing Strategy + +### Phase 1A Testing Focus +- Database operations and data consistency +- Service startup and health check reliability +- Mock authentication token flows +- Basic API functionality + +### Phase 1B Testing Focus +- Redis connectivity and session persistence +- Cache performance and invalidation +- Service behavior with Redis failures +- Session security and tampering protection + +### Phase 1C Testing Focus +- External authentication integration +- Token validation and refresh flows +- Role-based access across services +- System resilience under auth failures +- End-to-end user workflows + +## Benefits of Progressive Approach + +1. **Incremental Complexity**: Each phase adds one major component +2. **Isolation of Issues**: Problems can be traced to specific layers +3. **Thorough Testing**: Each layer gets comprehensive validation +4. **Rollback Capability**: Can revert to previous working state +5. **Learning Curve**: Understand each technology deeply before combining +6. **Debugging Ease**: Fewer variables when troubleshooting +7. **Confidence Building**: Success at each phase builds momentum + +## Deliverables Timeline + +- **Week 1**: Phase 1A - Backend + Database foundation +- **Week 2**: Phase 1B - Redis integration and testing +- **Week 2-3**: Phase 1C - Zitadel integration and full stack testing + +This progressive approach ensures solid foundations at each layer before adding the next level of complexity, making the overall implementation more reliable and maintainable. + +--- + +## Detailed Implementation Steps for Phase 1A + +### Step 1A.1: Create Go Project Structure + +#### Root Go Module Setup +```bash +# Initialize main go.mod +go mod init knowfoolery +go mod tidy +``` + +#### Directory Structure Creation +``` +knowfoolery/ +├── backend/ +│ ├── services/ +│ │ ├── game-service/ +│ │ │ ├── cmd/main.go +│ │ │ ├── internal/ +│ │ │ │ ├── handlers/ +│ │ │ │ ├── services/ +│ │ │ │ ├── middleware/ +│ │ │ │ └── models/ +│ │ │ ├── config/ +│ │ │ ├── tests/ +│ │ │ └── go.mod +│ │ ├── question-service/ +│ │ ├── user-service/ +│ │ ├── leaderboard-service/ +│ │ ├── session-service/ +│ │ ├── admin-service/ +│ │ └── gateway-service/ +│ ├── shared/ +│ │ ├── auth/ +│ │ ├── database/ +│ │ ├── observability/ +│ │ ├── security/ +│ │ └── utils/ +│ └── scripts/ +└── infrastructure/ + └── docker/ +``` + +### Step 1A.2: Define Ent Schemas + +#### Question Schema +```go +// backend/shared/database/ent/schema/question.go +package schema + +import ( + "time" + "entgo.io/ent" + "entgo.io/ent/schema/field" + "entgo.io/ent/schema/index" +) + +type Question struct { + ent.Schema +} + +func (Question) Fields() []ent.Field { + return []ent.Field{ + field.String("id").Unique(), + field.String("theme").NotEmpty(), + field.Text("text").NotEmpty(), + field.String("answer").NotEmpty(), + field.Text("hint").Optional(), + field.Enum("difficulty").Values("easy", "medium", "hard").Default("medium"), + field.Bool("is_active").Default(true), + field.Time("created_at").Default(time.Now), + field.Time("updated_at").Default(time.Now).UpdateDefault(time.Now), + } +} + +func (Question) Indexes() []ent.Index { + return []ent.Index{ + index.Fields("theme"), + index.Fields("difficulty"), + index.Fields("is_active"), + index.Fields("created_at"), + } +} +``` + +### Step 1A.3: Basic Service Implementation + +#### Game Service Main +```go +// backend/services/game-service/cmd/main.go +package main + +import ( + "log" + "github.com/gofiber/fiber/v2" + "github.com/gofiber/fiber/v2/middleware/cors" + "github.com/gofiber/fiber/v2/middleware/logger" +) + +func main() { + app := fiber.New(fiber.Config{ + AppName: "Know Foolery Game Service", + }) + + // Middleware + app.Use(logger.New()) + app.Use(cors.New()) + + // Health endpoint + app.Get("/health", func(c *fiber.Ctx) error { + return c.JSON(fiber.Map{ + "status": "healthy", + "service": "game-service", + }) + }) + + log.Println("Game Service starting on :3001") + log.Fatal(app.Listen(":3001")) +} +``` + +### Step 1A.4: Docker Compose Basic Setup + +#### Basic Docker Compose +```yaml +# infrastructure/docker/docker-compose.basic.yml +version: '3.8' + +services: + postgres: + image: postgres:15-alpine + environment: + POSTGRES_DB: knowfoolery + POSTGRES_USER: knowfoolery + POSTGRES_PASSWORD: dev-password + ports: + - "5432:5432" + volumes: + - postgres_data:/var/lib/postgresql/data + - ./init-db.sql:/docker-entrypoint-initdb.d/init-db.sql + healthcheck: + test: ["CMD-SHELL", "pg_isready -U knowfoolery"] + interval: 10s + timeout: 5s + retries: 5 + +volumes: + postgres_data: +``` + +### Step 1A.5: Integration Test Framework + +#### Test Utilities +```go +// backend/shared/database/testutil/testutil.go +package testutil + +import ( + "context" + "testing" + "github.com/testcontainers/testcontainers-go" + "github.com/testcontainers/testcontainers-go/modules/postgres" +) + +func SetupTestDB(t *testing.T) (*ent.Client, func()) { + ctx := context.Background() + + postgresContainer, err := postgres.RunContainer(ctx, + testcontainers.WithImage("postgres:15-alpine"), + postgres.WithDatabase("testdb"), + postgres.WithUsername("test"), + postgres.WithPassword("test"), + ) + if err != nil { + t.Fatal(err) + } + + // Get connection string and create Ent client + // ... implementation details + + cleanup := func() { + postgresContainer.Terminate(ctx) + } + + return client, cleanup +} +``` + +## Current Phase 1A Implementation Checklist + +- [ ] Create complete Go project structure +- [ ] Initialize all service modules with basic Fiber apps +- [ ] Define and generate all Ent schemas +- [ ] Set up database connection and migration utilities +- [ ] Implement mock JWT authentication +- [ ] Create Docker Compose basic environment +- [ ] Set up integration testing framework +- [ ] Add health checks and basic monitoring +- [ ] Create database seeding scripts +- [ ] Validate all services start and communicate properly + +This detailed plan provides clear, actionable steps for implementing Phase 1A while maintaining the progressive approach that allows for thorough testing at each layer. \ No newline at end of file