4. Execution RoadmapPhase 4: Keeper Network & Infra

4.4. Phase 4: Keeper Network & Infrastructure

Phase 4 transitions the Fhenix-FairMarket protocol from manual execution to a fully decentralized, self-sustaining automation network. While Phases 1-3 established the cryptographic and settlement foundations, Phase 4 guarantees system liveness and reliability by deploying a robust Keeper Network and production-grade infrastructure.

By converting closure triggers to public functions with economic incentives, implementing batch-optimized dispatch logic, and enforcing strict race-condition prevention via distributed locking, this phase ensures that auctions resolve deterministically without human intervention, while CI/CD pipelines maintain continuous security and quality assurance.

Phase Objective

  • Convert triggerFinalize() into a public function rewarding the first executor with a 0.2% bounty.
  • Build a resilient Keeper Network (auctionMonitor, cofheDispatcher, avsSubmitter) with retry and timeout logic.
  • Implement Batch Processing (max 10 auctions/block) to prevent Out-of-Gas (OOG) under network load.
  • Enforce Race Condition Prevention using Redis distributed locks, nonce checks, and blockhash salting.
  • Deploy a complete local simulation environment via Docker Compose and automate testing/deployment through GitHub Actions.

Target File Structure

packages/keeper/
├── src/
│ ├── index.ts # Service entry point & graceful shutdown handling
│ ├── config.ts # Network config, API keys, thresholds, batch limits
│ └── services/
│ ├── auctionMonitor.ts # WebSocket/Polling listener, endTime scheduler, retry logic
│ ├── cofheDispatcher.ts # Batch Queue, FHEOS routing, timeout handling (120s)
│ └── avsSubmitter.ts # EigenLayer signature aggregation, Fraud Proof validation

├── docker-compose.yml # Local stack: 3 Keepers + Redis + Prometheus/Grafana
├── .env.example # RPC_URL, FHEOS_API_KEY, EIGEN_LAYER_RPC, PRIVATE_KEY
└── package.json # Dependencies: ethers/viem, redis, node-cron, dotenv

.github/workflows/
├── ci-contracts.yml # Solidity compile, Slither, gas-reporter, unit tests
├── ci-keeper.yml # Keeper unit tests, Docker build verification
└── deploy-testnet.yml # Conditional deployment to Fhenix Sepolia after CI pass

️ Execution Tasks & Audit Gates

TaskDescriptionAudit Gate (P0)
4.1 Auction MonitorDevelop auctionMonitor.ts using WebSocket + fallback polling (30s). Detect endTime - 60s and queue for execution.Monitor detects closure within ±1 block. No auctions forgotten or missed.
4.2 CoFHE DispatcherBuild Batch Queue capping at 10 auctions/block. Route to FHEOS with API auth, 120s timeout, and exponential backoff retry.Processing never exceeds 10 auctions/block. Total latency ≤ 120s. Zero duplicate requests.
4.3 AVS SubmitterIntegrate avsSubmitter.ts to collect operator signatures, verify 3/5 threshold, aggregate proof, and call submitResolution().Contract accepts result only after valid AVS proof. Local Fraud Proof check prevents spoofing.
4.4 Race PreventionImplement Redis distributed locks, nonce tracking, and blockhash salting. Add dynamic priorityFee for execution priority.Zero duplicate transactions per auction. Distributed lock works flawlessly. Race rate ≤ 1%.
4.5 Docker StackConfigure docker-compose.yml with auctionMonitor + cofheDispatcher + avsSubmitter + Redis. Add .env.example template.docker compose up runs locally without errors. Full cycle simulatable from desktop.
4.6 CI/CD PipelinesConnect ci-contracts.yml (Slither, gas-reporter) and ci-keeper.yml to block merges on test failures.CI passes 100% before merge. Zero High/Critical Slither findings. Gas report auto-generated.

Technical Implementation

1. Public Finalization & Keeper Incentive (triggerFinalize)

The protocol incentivizes external actors to execute closure, guaranteeing liveness even if internal infrastructure fails.

// packages/contracts/core/FhenixFairMarket.sol
/**
 * @notice Public function to trigger auction finalization
 * @dev Rewards first valid executor with 0.2% of auction value
 * @param auctionId Target auction identifier
 */
function triggerFinalize(uint256 auctionId) external nonReentrant {
 require(auctions[auctionId].state == AuctionState.ACTIVE, "Invalid state");
 require(block.timestamp >= auctions[auctionId].endTime, "Auction not ended");
 
 // Race condition guard: blockhash or nonce check enforced here or via msg.sender tracking
 auctions[auctionId].state = AuctionState.RESOLVING;
 
 emit DecryptionRequested(auctionId, auctions[auctionId].ciphertextHashes);
 
 // Keeper bounty release logic (0.2%) implemented in settlement engine or direct transfer
 _distributeKeeperBounty(auctionId, msg.sender);
}

2. Batch Queue & Dispatcher Logic (cofheDispatcher.ts)

The dispatcher aggregates pending auctions and routes them securely to off-chain processors.

// packages/keeper/src/services/cofheDispatcher.ts
import Redis from 'ioredis';
import axios from 'axios';
 
const MAX_BATCH_SIZE = 10;
const FHEOS_TIMEOUT = 120_000; // 120 seconds
 
export class CoFHEDispatcher {
 private redis: Redis;
 
 async enqueueAndProcess(auctionIds: string[]) {
 for (const id of auctionIds) {
 await this.redis.rpush('pending_auctions', id);
 }
 
 // Process in batches
 while (await this.redis.llen('pending_auctions') > 0) {
 const batch = await this.redis.lrange('pending_auctions', 0, MAX_BATCH_SIZE - 1);
 
 try {
 const response = await axios.post(
 process.env.FHEOS_ENDPOINT,
 { auctionIds: batch },
 { headers: { 'Authorization': `Bearer ${process.env.FHEOS_API_KEY}` }, timeout: FHEOS_TIMEOUT }
 );
 // Queue AVS submission
 await this.avsSubmitter.handleResult(response.data);
 await this.redis.ltrim('pending_auctions', batch.length, -1);
 } catch (err) {
 console.error('FHEOS dispatch failed, retrying with backoff...', err);
 await this.retryWithBackoff(batch);
 break;
 }
 }
 }
}

3. Race Condition Prevention (Redis Distributed Lock)

To prevent multiple Keepers from triggering the same auction and wasting gas.

// packages/keeper/src/services/raceGuard.ts
import Redis from 'ioredis';
 
export class RaceGuard {
 constructor(private redis: Redis) {}
 
 async acquireLock(auctionId: string, ttl: number = 30): Promise<boolean> {
 const lockKey = `lock:auction:${auctionId}`;
 // Set lock only if not exists (atomic)
 const result = await this.redis.set(lockKey, 'locked', 'EX', ttl, 'NX');
 return result === 'OK';
 }
 
 async releaseLock(auctionId: string) {
 await this.redis.del(`lock:auction:${auctionId}`);
 }
}

Infrastructure Architecture

️ Security & Reliability Guarantees

  1. Decentralized Liveness: The 0.2% bounty ensures that even if internal Keepers fail, external actors are economically incentivized to call triggerFinalize(), preventing frozen auctions.
  2. OOG & Load Protection: The strict 10 auctions/block batch limit prevents gas exhaustion during network congestion spikes. Batches are processed sequentially with backoff, maintaining system stability.
  3. Race Condition Neutralization: Redis distributed locks and atomic nonce tracking guarantee that only one Keeper successfully triggers closure per auction. Failed attempts are silently dropped without gas waste.
  4. Resilient Dispatch Pipeline: cofheDispatcher implements exponential backoff retry logic and a hard 120s timeout. If FHEOS becomes unreachable, requests persist in the queue until resolution or until the Dynamic Dead Man's Switch safely voids the auction.
  5. Continuous Security Enforcement: ci-contracts.yml automatically runs Slither and Mythril on every PR. High/Critical findings block merges immediately, preventing vulnerable code from reaching production.

Phase 4 Audit Gate (P0 Checklist)

Progression to Phase 5 is strictly blocked until all P0 criteria are verified and merged:

  • [] Keepers auto-invoke triggerFinalize() at endTime without human intervention or missed auctions.
  • [] Batch processing never exceeds 10 auctions/block and maintains stable gas consumption under simulated load.
  • [] docker compose up successfully launches all services locally, simulating a full execution cycle.
  • [] Race condition prevention works flawlessly: ≤ 1% duplicate execution rate in stress tests.
  • [] CI/CD pipelines (ci-contracts.yml, ci-keeper.yml) pass 100% with zero High/Critical security findings.
  • [] avsSubmitter correctly rejects results below the operator threshold (e.g., 3/5).

Timeline & Dependencies

MetricValue
Estimated Duration10–12 days
Team Size2–3 Engineers (DevOps + TypeScript + Solidity)
DependenciesPhase 3 (Keeper interfaces, AVS mocks, CoFHE flow)
EnablesPhase 5 (Frontend real-time tracking), Phase 6 (Testnet deployment & monitoring)

Parallel Development Note: Phase 5 (Frontend UX 2.0) can begin scaffolding using mock ABIs, but full integration with real-time status dashboards requires Phase 4’s WebSocket and event emission architecture to be finalized.


Next Steps