🚀 go-pugleaf

Welcome to our new high-performance NNTP newsgroup system built with Go.

**🚀 ANNOUNCEMENT: First go-pugleaf NNTP Node Online - Carrying Forward Retro Guy's Legacy**

We are proud to announce that the first **go-pugleaf** NNTP node is now online and operational!

**🌐 Access Information:**
- **Web Interface:** pugleaf.net
- **NNTP Server:** Coming within the next few days or weeks
- **Coverage:** Full rocksolid.* hierarchy + additional groups

**📊 Current Status:**
✅ **ONLINE** - Web interface fully functional
🔄 **BACKFILLING** - Importing articles from legacy rslight installations
📥 **FETCHING** - Automated article retrieval via cron jobs

**🛠️ Development Roadmap:**
- **Next Few Days:** NNTP server for reading access
- **In Progress:** Incoming article peering setup
- **TODO:** Posting access and full bidirectional peering

**💝 Tribute to Retro Guy:**
This project carries forward the legacy of **Retro Guy** @novabbs.com
The visionary lead developer of rslight.
Though he sadly cannot see this evolution of his work... RIP!
We are committed to keeping the rocksolid.* community alive and thriving.

The fresh, modern interface and robust Go-based architecture represent
our dedication to honoring his contributions while building for the future.

**🔧 Technical Highlights:**
- Built with modern Go architecture for performance and reliability
- High-speed parallel article fetching and processing
- Efficient database design for large-scale article storage
- Full NNTP protocol compliance

**🤝 Community Notes:**
We're in the exciting phase of bringing historical rocksolid.* content back online!
The migration from legacy rslight installations is proceeding smoothly, ensuring no community history is lost.

**🎯 Mission:**
To provide a stable, modern, and efficient platform for the rocksolid.* community and beyond!

Stay tuned for NNTP access announcements and peering updates!

**The go-pugleaf Development Team**
*Honoring the past, building the future*

---
                    

### Core NNTP Server Architecture
- [x] **Server Foundation**:
- [x] Main NNTP server struct with configuration
- [x] TCP listener with TLS/SSL support
- [x] Client connection management
- [x] Connection pooling and resource limits
- [x] Graceful shutdown handling
- [x] Multi-threaded request processing

### NNTP Protocol Implementation (RFC 3977)
- [x] **Essential Commands (Priority 1)**:
- [x] `CAPABILITIES` - Server capability announcement
- [x] `MODE READER` - Switch to reader mode
- [x] `AUTHINFO USER/PASS` - User authentication
- [x] `QUIT` - Clean connection termination
- [x] `HELP` - Command help system

- [x] **Group Management Commands**:
- [x] `LIST` - List available newsgroups
- [x] `LIST ACTIVE` - Active groups with counts
- [x] `LIST NEWSGROUPS` - Groups with descriptions
- [x] `GROUP` - Select newsgroup and get stats
- [x] `LISTGROUP` - List article numbers in group

- [x] **Article Retrieval Commands**:
- [x] `STAT` - Check article existence
- [x] `HEAD` - Retrieve article headers
- [x] `BODY` - Retrieve article body
- [x] `ARTICLE` - Retrieve complete article
- [x] Support for both message-ID and article number addressing

- [x] **Overview Commands**:
- [x] `XOVER` - Article overview data (NOV format)
- [x] `XHDR` - Specific header retrieval
- [ ] `HDR` - RFC 3977 header command
- [ ] `OVER` - RFC 3977 overview command

- [ ] **Advanced Commands (Priority 2)**:
- [x] `POST` - Article posting (optional)
- [x] `IHAVE` - Article transfer (server-to-server)
- [x] `CHECK/TAKETHIS` - Streaming commands
- [ ] `DATE` - Server date/time
- [ ] `NEWGROUPS` - New groups since date
- [ ] `NEWNEWS` - New articles since date

### Database Integration
- [x] **Data Access Layer**:
- [x] Integrate with existing `*database.Database`
- [x] Reuse `models.Article`, `models.Newsgroup` structures
- [x] Efficient article retrieval by message-ID and number
- [x] Group statistics and article counting
- [x] XOVER data generation from database
- [x] Range-based overview queries for XOVER/XHDR
- [x] Article caching for frequently accessed content

### Authentication & Security
- [x] **User Authentication**:
- [x] Integrate with existing user system (`models.User`)
- [x] AUTHINFO command with database user validation (framework)
- [x] Session management for authenticated connections
- [x] Permission-based group access control

- [ ] **Security Features**:
- [ ] Rate limiting per connection/IP
- [ ] Command validation and sanitization
- [ ] Logging of all NNTP operations
- [ ] Connection timeout management
- [ ] IP-based access control (optional)

### Configuration & Management
- [x] **Server Configuration**:
- [x] NNTP server port and binding configuration
- [x] TLS/SSL certificate management
- [x] Maximum connections and resource limits
- [ ] Command-specific feature toggles
- [x] Integration with existing `config.ServerConfig`

- [ ] **Administration Integration**:
- [ ] NNTP server status in admin dashboard
- [ ] Real-time connection monitoring
- [ ] Command usage statistics
- [ ] Server performance metrics
- [ ] Configuration management via web interface

### Testing & Validation
- [ ] **Protocol Compliance**:
- [ ] RFC 3977 compliance testing
- [ ] Test with standard NNTP clients (tin, slrn, Thunderbird)
- [ ] Multi-line response handling
- [ ] Error code consistency
- [ ] Character encoding support (UTF-8)

---
                    

### Admin Interface
- [x] Create admin dashboard
- [x] API Tokens Management
- [ ] Add server status monitoring
- [x] Implement newsgroup management:
- [x] Add new groups
- [x] Delete groups
- [x] Modify group settings
- [x] sections Create/Set/Modify for newsgroups
- [x] Bulk operations
- [x] Create user management interface
- [x] Add provider configuration
- [x] Implement system settings
- [ ] Create backup/restore tools
- [ ] Implement backup automation

### Monitoring & Logging
- [ ] Create health check endpoints
- [ ] Create log rotation
- [ ] Add alerting system

### Maintenance Tools
- [x] Create article cleanup tools
- [x] Add database optimization
- [x] Create migration scripts

---
                    

### User Management ✅ COMPLETE
- [x] Implement user registration with validation
- [x] Add password hashing (bcrypt)
- [x] Create login/logout system
- [x] Implement session management
- [x] Add JWT token support
- [x] Create user profile system
- [x] Profile: password change functionality
- [ ] Add password reset functionality via email
- [ ] Add external SMTP sending support
- [ ] Implement email verification for new accounts (+ option in admin panel do enable/disble)
- [x] Implement user roles/permissions
- [x] Add user registration form template
- [x] Create login/logout page templates
- [x] **Admin interface for user management (create/update/delete)**
- [x] **API token management system with CRUD operations**
- [x] **Session-based web authentication with secure cookies**

### Search Functionality ⚡ IN PROGRESS
- [x] **Basic search page template implemented**
- [x] **Search API endpoints (newsgroup search working)**
- [x] Create search API endpoints
- [x] **Newsgroup name search functionality**
- [ ] Implement full-text search for articles (FTS5 schema ready)
- [x] Add search page template
- [x] Add advanced search filters (author, date, group)
- [x] Implement search result pagination
- [ ] Add search result highlighting

### Security Features ✅ PARTIALLY COMPLETE
- [x] **API token authentication system with secure hashing**
- [x] **Secure session handling with HttpOnly cookies**
- [x] **Password validation and bcrypt hashing**
- [x] **Input validation framework**
- [x] **SQL injection prevention (parameterized queries)**
- [x] **XSS protection via template HTML escaping**
- [x] Implement SQL injection prevention (already in place with parameterized queries)
- [ ] Add CSRF protection
- [ ] Implement rate limiting middleware
- [ ] Add comprehensive input validation framework
- [ ] Add file upload security
- [ ] Create audit logging

### Theme & UI Enhancements ✅ COMPLETE
- [x] **Bootstrap 5.3 dark mode integration**
- [x] **Modern theme switcher with localStorage persistence**
- [x] **Responsive design optimized for mobile**
- [x] **Professional navigation with authentication links**
- [x] **Theme selector positioned bottom-right to avoid overlap**
- [x] **Page refresh functionality for complete theme application**

---
                    

### Gin Web Framework Setup
- [x] Set up Gin web server (internal/web/)
- [x] Create router configuration
- [x] Add paginated API endpoints:
- [x] GET /api/v1/groups (list newsgroups)
- [x] GET /api/v1/groups/:group/overview (group overview)
- [x] GET /api/v1/groups/:group/articles/:articleNum (single article)
- [x] GET /api/v1/groups/:group/threads (thread data)
- [x] Create cmd/web/main.go server entry point
- [x] Add middleware:
- [x] Logging middleware
- [x] Static file serving
- [x] CORS middleware
- [x] Error handling middleware
- [x] Create comprehensive error handling system
- [x] Add request validation

### 🔥 REVOLUTIONARY FORUM-STYLE THREADING SYSTEM
- [x] **Clean thread list page**: Shows only thread titles with message counts
- [x] **Full thread view route**: `/groups/:group/thread/:threadRoot`
- [x] **Complete forum experience**: Click thread → see full discussion
- [x] **Full article content inline**: All messages display complete article bodies
- [x] **Flat threading hierarchy**: Forum-style (no confusing nested replies)
- [x] **Smart threading engine**: Parses Message-ID/References for perfect grouping
- [x] **Chronological message sorting**: Root first, then replies by date
- [x] **Rich metadata display**: Author, date, lines, bytes for each message
- [x] **Thread navigation**: Clean breadcrumbs and back-to-threads links

### HTML Templates & UI
- [x] Create base template layout with theme support
- [x] Design newsgroup listing page with pagination
- [x] Create article display template with proper formatting
- [x] Implement pagination controls for Groups and Articles
- [x] **Revolutionary thread.html template with full article content**
- [x] **Updated threads.html for clean thread-only listing**
- [x] Create stats page with system information
- [x] Add help page with documentation
- [x] Design error page with user-friendly messages
- [x] Add comprehensive navigation
- [x] **Forum-style thread discussions with inline content**

### Static Assets & Frontend
- [x] Create modern CSS framework (responsive design)
- [x] Implement multiple theme system:
- [x] Modern theme (gradient design with animations)
- [x] Classic theme (retro terminal green-on-black)
- [x] Amber theme (vintage amber terminal)
- [x] Add JavaScript functionality:
- [x] Theme switching with localStorage persistence
- [x] Responsive navigation
- [x] Pagination controls
- [x] Article navigation
- [x] Mobile-friendly responsive design
- [x] RockSolid rocket logo integration

### Performance & Scalability
- [x] Implement efficient pagination for all large datasets
- [x] Optimize database queries with proper indexing
- [x] Memory-efficient template rendering
- [x] Preloader system for newsgroup descriptions
- [x] Reply count tracking and display
- [x] Error handling for missing articles/groups
- [x] **Full article loading optimization for thread views**
- [x] **Efficient thread message grouping and sorting**

---
                    

### Database Schema Design
- [x] Design main database schema
- [x] config table (system configuration)
- [x] providers table (NNTP server configurations)
- [x] newsgroups table (subscribed groups metadata)
- [x] users table (user accounts)
- [x] sessions table (user sessions)
- [x] user_permissions table (access control)
- [x] Design per-group database schemas
- [x] articles table (full article storage)
- [x] **threads table (revolutionary forum-style threading)**
- [x] overview table (XOVER data for performance)
- [x] Create database migration system
- [x] Implement database connection pooling

### SQLite3 Implementation
- [x] Create database abstraction layer (internal/database/)
- [x] Implement main database management
- [x] Implement per-newsgroup database management
- [x] Add article storage functions
- [x] **Implement revolutionary threading logic using Message-ID/References**
- [x] **Forum-style flat threading (one thread per original post)**
- [x] Create overview data management (XOVER caching)
- [x] Add database indexing for performance
- [x] Implement efficient batching and scalability features
- [x] **Thread counting and message grouping**
- [ ] Implement database backup/restore

### Data Models & Import System
- [x] Create Go structs for all data models
- [x] Implement JSON serialization
- [x] Add validation tags
- [x] Create model relationships
- [x] Add database query methods
- [x] Implement NNTP-to-database import pipeline:
- [x] Group subscription and metadata import
- [x] Article batch import using XOVER/XHDR
- [x] Threading logic based on Message-ID/References
- [x] Incremental sync for new articles
- [x] Downloaded article tracking system
- [x] Scalable batching for large groups
- [x] Batched sync for new articles
- [x] Downloaded article tracking system
- [x] Scalable batching for large groups

---