A robot can recognize a cup in a photograph and still fail to pick it up. The image may show what the cup looks like, but the robot also needs to estimate its size, position, distance and grip. It must then move its arm without knocking over nearby objects. Hesai’s Kosmo is designed to help developers build that kind of spatial understanding by turning real places into detailed 3D environments that AI systems can use for training.
Kosmo is a system rather than a camera alone. Its Kosmo One device combines lidar, an 8K imaging system and an inertial measurement unit, which tracks the camera’s movement and orientation. The sensors capture geometry, colour, texture and position together. Hesai says a typical 100-square-metre indoor space can be scanned in about 15 minutes, with the resulting data processed by Kosmo World, the company’s cloud platform.
From a Scan to a Training Environment
Kosmo World reconstructs a captured space in several forms. One is 3D Gaussian Splatting, a way of representing a scene with many small, view-dependent elements that can reproduce its appearance from different angles. The system also generates meshes, which describe surfaces in a form that software can interact with, and depth data, which records how far objects are from the camera.
Together, these outputs can give a simulation more to work with than a collection of flat images. A robot’s training environment can include visual detail, object shapes and spatial relationships, allowing developers to practise tasks such as navigating around furniture or reaching for items on a shelf. Kosmo World can also identify individual objects and generate additional views designed to reflect a robot’s own cameras.
Hesai says its software can fill in areas missed during scanning and reconstruct materials and textures. That may help create a more complete scene, but generated detail is still an estimate. If a robot’s training environment quietly invents a handle, gap or surface feature, the robot may learn the wrong lesson. Developers will need to check how well a reconstructed scene matches the place where the robot is expected to work.
Why More Realistic Training Matters
Robots often train in simulation because it is faster and safer than collecting every example in the physical world. But simulations can leave out the clutter, lighting changes and awkward object positions found in real homes, shops and factories. The mismatch between a training environment and the physical world can make a capable-looking robot unreliable once it encounters something slightly different.
Hesai’s pitch is that Kosmo can make it easier to build training environments from real spaces. The company says a deployment at a Dairy Queen store reduced the time needed to create a 3D training environment from around 300 hours of manual modelling to a 15-minute scan and reconstruction process. It also reports that a Sharpa robot achieved a 99% task success rate in that deployment. Those figures are company-reported results from a specific setting; they do not establish how the system would perform across other stores, tasks or changing conditions.
Realistic training data could help robots practise more variations before they encounter a task on the job. Kosmo’s world model pipeline can also generate training trajectories from a captured scene. This could let developers test different routes, viewpoints or interactions without staging each one physically. The benefit depends on whether the simulated situations capture the important ways a task can go wrong.
Seeing Is Only Part of the Job
A detailed map does not tell a robot what to do. The robot still needs software that can interpret instructions, decide which objects matter, plan a safe sequence of movements and adjust when something changes. Its sensors must keep up with people walking through the space, products being moved, doors closing and objects behaving differently than expected.
Physical actions add another layer of difficulty. A robot may locate an ice-cream container accurately but still struggle to grasp it, control its force or recover if it slips. That requires capable motors and hands, reliable perception during movement, and systems that can detect mistakes and respond safely. Mapping can improve what a robot starts with; it cannot guarantee what the robot will do next.
Kosmo’s role is therefore best understood as part of a broader robotics toolkit. It can help convert a real environment into reusable data for simulation and AI training, while the robot’s perception, planning and motion systems must turn that preparation into dependable behaviour. The closer the training environment is to the real one, the more useful it may be—but the robot still has to prove itself outside the scan.
Final Byte
Hesai is applying its lidar experience to a problem that sits upstream of robot deployment: giving developers better records of the spaces where machines are expected to work. Kosmo could make those environments quicker to capture and easier to use in training. Its value will ultimately be measured by whether robots can carry what they learn into unfamiliar moments, where a small change in the room can become a very real test.
