Warehouse Automation & Order Picking
Reinforcement learning and swarm optimization for multi-agent order picking
Overview
Order picking is the single most time-consuming operation in warehouse logistics. This project treats it as a multi-agent sequential decision problem: given a warehouse layout, a product catalog and a set of customer orders, several picking vehicles must be routed to collect every item and return to the exit in the shortest total time, without colliding in shared aisles. The work targets steel warehouses, where products are long profiles stored across multiple slots and a single order may span dozens of locations.
Learning and Optimization Approaches
Two families of algorithms are implemented against a common simulation environment so their results can be compared directly on the same layouts and order sets. The first is swarm-based: a time-dependent Ant Colony Optimization planner that builds a pheromone matrix over the warehouse grid and resolves collisions by treating occupancy as a function of time rather than space alone. The second is deep reinforcement learning — DQN, Dueling DQN, Double Dueling DQN and PPO agents, implemented in PyTorch, which learn a picking policy through repeated episodes instead of planning each route from scratch.
Simulation Environment
Warehouses are described as JSON grid maps defining dimensions, aisles, loading zones, entrances, exits and item locations. The reference layout models a real steel warehouse: a 30 × 28 grid holding 289 distinct product types, each stored across several slots and linked to live order identifiers. A smaller 12 × 7 layout is used for rapid iteration during training. The environment exposes a standard step-and-reward interface, so agents and classical planners can be swapped without changing the warehouse model.
Simulation Platform
The research code is wrapped in a browser-based platform that makes experiments repeatable without touching Python. Operators upload a warehouse map, choose a planning algorithm, set training hyperparameters — episode count, learning rate, discount factor, epsilon schedule, batch size — and either train a fresh policy or run an existing one. Simulation runs are rendered back as video for inspection.
Two-Sided Workflow
The platform separates the people who define a warehouse from the people who order from it. On the operator side, warehouse maps and product catalogs are uploaded and managed directly or pulled from an existing endpoint. On the ordering side, a guided flow walks through selecting a warehouse, building an order from the catalog, choosing an algorithm and watching the resulting simulation.
Current Status
The ACO planner runs against production-scale layouts, and the reinforcement learning agents train and execute on the same environment. Work continues on comparative benchmarking across layouts and order profiles, and on packaging the platform for a public demonstration deployment.