Unity Formula Student Driverless Simulator
The UBM Formula Student Driverless Simulator is a Unity-based 3D front end for
the ubm-fsae ROS 2 stack. It loads cone tracks, renders the Formula Student
vehicle and environment, receives the vehicle pose from ROS, and publishes
simulated LiDAR, stereo-camera, cone, and transform data.
The simulator can also run without ROS. In standalone mode, the vehicle can be driven with a keyboard, gamepad, or joystick, which is useful for inspecting a track, validating sensor settings, and measuring simulator performance before connecting the complete pipeline.
Typical uses include:
- visualizing the vehicle on Formula Student cone tracks;
- testing the perception pipeline with simulated point clouds and stereo camera data;
- receiving localization output from
/ekf_pose/odomand applying it to the simulated vehicle; - checking TF, sensor frames, topic names, rates, and payload sizes;
- evaluating different LiDAR sampling and stereo-camera configurations;
- reproducing tests with bundled TrackGen/NavSim maps;
- running manual driving sessions, cone-impact tests, and timed laps.
UNITY_SIM_SCREENSHOT_01 — Simulator overview
Replace this note with a screenshot of the running simulator showing the
vehicle, track, cone counter, session timer, Settings button, and diagnostic
HUD. Suggested file: docs/assets/unity-simulator/runtime-overview.png.
Use: 
How the simulator fits into the ROS pipeline
The Unity simulator supports two communication transports:
- ROS-TCP-Connector, the default transport, using a ROS-TCP-Endpoint on the ROS 2 computer;
- ROS#, the fallback transport, using rosbridge over a WebSocket.
Both transports expose the same simulator topics to the rest of the stack.
odometry input
ROS 2 ------------------------------------> Unity vehicle pose
/ekf_pose/odom
ROS 2 <------------------------------------ Unity simulated sensors
/lidar_points, /camera_data, /cones, /tf
Unlike NavSim, the Unity simulator does not publish simulated odometry. In ROS odometry mode it subscribes to odometry and uses it to position the vehicle. It then produces sensor data from that pose and the 3D scene.
The simulator does not currently calculate complete vehicle dynamics. Standalone movement is kinematic, and ROS mode follows received odometry. Cone collisions use lightweight Unity rigidbodies and are not a validated Formula Student impact model.
Requirements
- Unity Hub;
- Unity Editor
6000.6.0f1; - Git and Git LFS;
- network access while Unity resolves packages from Git;
- for ROS integration, an Ubuntu 24.04 system running ROS 2 Jazzy and a built
ubm-fsaeworkspace.
The required Unity version is recorded in the project's
ProjectSettings/ProjectVersion.txt. Do not downgrade the project to the old
6000.0.32f1 version. The current version is newer than the fixed Unity
baseline for CVE-2025-59489.
Download and open the project
Install and initialize Git LFS before cloning. Vehicle FBX and Blender files are stored through LFS:
git lfs install
git clone https://github.com/ubm-driverless/ubm-fsae-sim.git
cd ubm-fsae-sim
git lfs pull
Avoid downloading the project as a GitHub source ZIP: depending on the archive configuration, large model files may be downloaded as small LFS pointer files instead of real assets.
Open Unity Hub and:
- install Unity
6000.6.0f1if it is not already available; - select Add -> Add project from disk;
- choose the cloned
ubm-fsae-simdirectory; - open the project with Unity
6000.6.0f1; - wait for Unity Package Manager and asset import to finish;
- open
Assets/Scenes/MainMenu.unity; - enter Play mode.
The project includes ROS-TCP-Connector, ROS#, URP, the Input System, and the
other Unity packages in Packages/manifest.json. They are resolved when the
project is opened for the first time.
If an FBX or .blend file contains only text beginning with
version https://git-lfs.github.com/spec/v1, close Unity and run:
UNITY_SIM_SCREENSHOT_02 — Unity project and Play button
Replace this note with a screenshot of Unity showing the Project panel,
Assets/Scenes/MainMenu.unity, and the Play button. Suggested file:
docs/assets/unity-simulator/unity-editor-start.png.
Use: 
Start a standalone simulation
ROS is optional for a standalone session.
- Open
Assets/Scenes/MainMenu.unityand enter Play mode. - Select an Athena or Visione vehicle.
- Select the display LOD. LOD 1 is the recommended default.
- Select the number of LiDAR vertical layers and points per layer.
- Choose one of the bundled tracks.
- Choose the asphalt preset. Time-of-day options are currently saved placeholders and do not yet provide a complete environment change.
- Click Start Simulation.
- Press
Esc, open Controls, and select Standalone input. - Drive with
W/A/S/Dor the arrow keys.
Standalone input uses simplified kinematic movement. It is intended for simulator interaction and sensor testing, not vehicle-dynamics validation.
Connect ROS 2
The transport is selected from Settings -> ROS connection after the simulation starts.
ROS-TCP-Connector
ROS-TCP-Connector is the default transport. On the ROS 2 computer:
- install ROS-TCP-Endpoint in the same workspace as
ubm_interfaces; - build the workspace;
- source ROS 2 and the workspace;
- start ROS-TCP-Endpoint on a network interface reachable from the computer
running Unity, using port
10000by default.
Follow the ROS 2 setup instructions in the official ROS-TCP-Endpoint repository for the exact installation and launch commands supported by the installed version.
In Unity:
- press
Esc; - open ROS connection;
- select ROS-TCP-Connector;
- set ROS-TCP host/IP to the ROS computer's address;
- keep port
10000unless the endpoint uses another port; - click Retry connection;
- check the connection status at the top of the page.
127.0.0.1 works only when Unity and ROS-TCP-Endpoint share the same host
network. When Unity runs on Windows and ROS runs on another Ubuntu computer,
enter the Ubuntu computer's LAN address.
ROS# and rosbridge fallback
ROS# communicates through rosbridge. In a terminal where ROS 2 and the built
ubm-fsae workspace are sourced, start rosbridge:
source /opt/ros/jazzy/setup.bash
cd /path/to/your/ubm-fsae-workspace
source install/setup.bash
ros2 launch rosbridge_server rosbridge_websocket_launch.xml port:=9090
In Unity:
- open Settings -> ROS connection;
- select ROS# fallback;
- set the rosbridge URL, for example
ws://localhost:9090orws://<ROS_HOST_IP>:9090; - click Retry connection.
The workspace containing ubm_interfaces must be sourced before starting
rosbridge; otherwise the bridge cannot advertise the custom camera message.
UNITY_SIM_SCREENSHOT_03 — ROS connection page
Replace this note with screenshots of the ROS-TCP and ROS# transport
selections, including endpoint fields and connected status. Suggested
file: docs/assets/unity-simulator/ros-connection.png.
Use: 
Select ROS odometry control
Connecting a transport does not automatically change the vehicle input mode. Open Settings -> Controls and select ROS odometry. The default input is:
| Topic | Type | Purpose |
|---|---|---|
/ekf_pose/odom |
nav_msgs/msg/Odometry |
UBM FSAE vehicle pose |
The ROS connection page provides two shortcuts:
- UBM FSAE topic selects
/ekf_pose/odom; - Legacy sim topic selects
/command_odom.
Verify the selected input on the ROS system:
Published ROS interfaces
Defaults are loaded from Assets/StreamingAssets/default_settings.json and can
be changed at runtime.
| Direction | Default topic | Message | Purpose |
|---|---|---|---|
| Subscribe | /ekf_pose/odom |
nav_msgs/msg/Odometry |
Vehicle position and orientation |
| Publish | /lidar_points |
sensor_msgs/msg/PointCloud2 |
Simulated LiDAR point cloud |
| Publish | /cones |
visualization_msgs/msg/MarkerArray |
Visible/noisy/missing cone simulation |
| Publish | /camera_data |
ubm_interfaces/msg/CameraData |
Left/right images and camera information |
| Publish | /tf |
tf2_msgs/msg/TFMessage |
Vehicle and sensor transforms |
The point cloud is normally published in the hesai_lidar frame. Vehicle and
sensor frame names can be edited from ROS connection -> Frames.
Useful checks include:
ros2 topic hz /lidar_points
ros2 topic hz /camera_data
ros2 topic echo /tf --once
ros2 topic echo /cones --once
The optional DriveSubscriber understands ubm_interfaces/msg/Control on
/control, but it is currently diagnostic. Vehicle movement in ROS mode still
follows odometry.
Launch menu
The launch menu configures the session before the simulation scene is loaded.
Vehicle and display LOD
- Athena and Visione select the vehicle model and matching sensor arrangement.
- LOD 0 High uses the most detailed display mesh.
- LOD 1 Medium is the default and recommended general-purpose choice.
- LOD 2 Low reduces rendering cost on slower computers.
The collision mesh remains separate from the selected visual LOD.
LiDAR sampling
- Vertical layers controls the number of simulated scan layers.
- Points per layer controls horizontal sampling.
- The summary displays the total points generated per scan.
Dense scans increase CPU work, message size, serialization cost, and network traffic. Start with a smaller configuration when testing on a low-spec laptop.
Track and environment
The track selector shows a top-down preview, cone counts, dimensions, format,
and start position. Bundled JSON and CSV maps are read from
Assets/StreamingAssets/.
The environment section selects an asphalt material. Time-of-day selections are stored for future use but are currently placeholders.
UNITY_SIM_SCREENSHOT_04 — Launch menu
Replace this note with a screenshot of the complete launch menu, including
vehicle, LOD, LiDAR sampling, track preview, and environment settings.
Suggested file: docs/assets/unity-simulator/launch-menu.png.
Use: 
Runtime Settings menu
Press Esc or click Settings to open the runtime panel. Vehicle input and
camera interaction are paused while the panel is open. Values are saved in
Unity PlayerPrefs when the panel closes.
Overview
Overview summarizes the ROS connection, active camera, graphics status, vehicle position, and diagnostics. It also links to maps, controls, ROS, and the main launch menu.
Maps
Maps searches and loads bundled tracks without restarting Play mode. It can also load a CSV or TrackGen/NavSim JSON file using an absolute filesystem path. Selecting a map does not activate it until Load selected map is pressed.
Controls
Controls switches between standalone input and ROS odometry and configures maximum speed, acceleration, and steering rate for standalone motion.
ROS connection
ROS connection selects the transport and endpoint, retries or closes the connection, edits topic names, selects odometry presets, configures TF frequency, and changes vehicle/sensor frame names.
Sensors
Sensors enables or disables the point cloud, cone, and stereo-camera publishers. Disabling unused publishers is the fastest way to reduce simulator load during movement-only tests.
The point-cloud section controls:
- topic and field/layer definition files;
- horizontal angular resolution and vertical-layer count;
- maximum distance and scan frequency;
- visualization of rays that hit cones.
The cone-detection section controls topic, mesh markers, distance, frequency, missing detections, noisy detections, and maximum position noise.
The stereo-camera section controls topic, publication frequency, image size, raw or JPEG payload, and JPEG quality. The available per-eye profiles are 1280×720, 960×540, 640×360, and 426×240.
UNITY_SIM_SCREENSHOT_05 — Sensor settings
Replace this note with a screenshot of the Sensors page showing publisher
toggles, LiDAR parameters, and stereo-camera profiles. Suggested file:
docs/assets/unity-simulator/sensor-settings.png.
Use: 
Camera
The simulator provides four display views:
- Chase follows the vehicle from behind;
- Overhead follows the position while keeping a fixed world orientation;
- Orbit rotates around and follows the vehicle;
- Free roam detaches the camera for independent scene inspection.
The page also controls overhead height/framing, orbit distance, orbit-axis inversion, and follow smoothing.
Graphics
Graphics provides Low, Balanced, and High runtime profiles, along with
target FPS, VSync, URP render scale, real-time shadows, and the performance HUD.
Graphics API selection requires restarting Unity; on Linux, use
-force-vulkan if the OpenGL editor backend produces rendering artifacts.
UNITY_SIM_SCREENSHOT_06 — Runtime Settings tabs
Replace this note with an annotated screenshot identifying Overview, Maps,
Controls, ROS connection, Sensors, Camera, and Graphics. Suggested file:
docs/assets/unity-simulator/runtime-settings-tabs.png.
Use: 
Tracks and map formats
Bundled tracks belong in Assets/StreamingAssets/. Supported files appear in
both the launch menu and the runtime map library and are included in standalone
builds.
The simulator supports:
- TrackGen/NavSim JSON;
- typed CSV with
start,blue,yellow,orange, andbigrows; - legacy
x,yCSV files, loaded as unknown cones.
A typed CSV example is:
TrackGen/NavSim JSON preserves initialPosition, cones, path, and significant
positions. Unity currently instantiates the cones and applies the initial pose;
path and significant-position data are accepted for compatibility but are not
used for Unity path following.
Unity uses x/z as the ground plane, while TrackGen and ROS use x/y:
| TrackGen/ROS | Unity |
|---|---|
x |
x |
y |
z |
| positive yaw | negative Unity yaw around y |
Press R to place the vehicle at the active map's start pose. If the map has
no start pose, the simulator uses the scene's initial vehicle position.
UNITY_SIM_SCREENSHOT_07 — Map library and track
Replace this note with a composite image showing a map selected in the
runtime library and the corresponding cone track in the scene. Suggested
file: docs/assets/unity-simulator/map-library.png.
Use: 
Driving, cone impacts, and run timing
In standalone mode, W/S controls throttle and brake/reverse, while A/D
steers. Arrow keys are aliases. Gamepads and generic joysticks are supported.
Track cones can be knocked over. Each cone impact is counted once. Press
Shift+R or use the cone HUD reset button to restore every cone and clear the
counter.
Press T to start or stop the run timer. The timer uses unscaled Unity time,
so it continues to measure consistently across frame-rate changes.
Cameras
Press C to cycle through Chase, Overhead, Orbit, and Free roam views.
Overhead
Use the mouse wheel to change altitude and orthographic framing. With a gamepad, the vertical left stick changes height; use the triggers for vehicle throttle while this camera is active.
Orbit
Use the mouse wheel to change distance. Drag with the left mouse button or use the gamepad right stick to orbit around the vehicle.
Free roam
Drag with either mouse button to look around. Use W/A/S/D to move on the
horizontal plane, E/Q to move vertically, and hold Shift for precise slow
movement. Vehicle keyboard input is disabled while Free roam is active.
Keyboard and mouse shortcuts
| Input | Action |
|---|---|
Esc |
Open or close runtime Settings |
C |
Cycle Chase, Overhead, Orbit, and Free roam cameras |
W/A/S/D or arrows |
Drive in standalone mode |
R |
Respawn at the active map start pose |
Shift+R |
Restore all cones and clear the impact counter |
T |
Start or stop the session timer |
F3 |
Show or hide the performance/ROS diagnostics HUD |
L |
Show or hide LiDAR rays that hit cones |
| Mouse wheel | Change Overhead height or Orbit distance |
| Left-mouse drag | Rotate the Orbit camera |
| Either-mouse drag | Look around in Free roam |
W/A/S/D in Free roam |
Move the detached camera horizontally |
E/Q in Free roam |
Move the detached camera up/down |
Shift in Free roam |
Precise slow camera movement |
Alt+F4 |
Close a standalone Linux/Windows player |
Vehicle and free-roam input are paused while Settings is open. Shortcut handlers also ignore focused text fields where applicable.
Performance and diagnostics
Press F3 to open the diagnostics HUD. It reports:
- FPS, frame time, and Unity/managed memory;
- LiDAR and camera publish cadence, payload size, and transmit age;
- odometry receive and application counts;
- odometry message age and recent message intervals;
- source motion and vehicle tracking error;
- vehicle ground clearance.
If performance is poor:
- disable unused publishers in Sensors;
- reduce LiDAR layers, points per layer, or scan frequency;
- reduce stereo resolution or frequency;
- enable JPEG payloads when network bandwidth is the bottleneck;
- select the Low or Balanced graphics profile;
- reduce URP render scale or disable real-time shadows.
Raw stereo images and dense point clouds are expensive to render, serialize, and transmit. The camera publisher keeps at most one stereo capture in flight and drops missed periods instead of building an unbounded queue.
Build a standalone player
In Unity, open File -> Build Profiles and create or select the target platform profile. The scene order must contain:
Assets/Scenes/MainMenu.unity;Assets/Scenes/scene.unity.
Build into a directory outside Assets/. Generated Unity directories such as
Library, Temp, Logs, Builds, and UserSettings must not be committed.
Troubleshooting
Unity cannot connect to ROS
- Confirm that the selected transport matches the server running on the ROS computer.
- For ROS-TCP, verify the endpoint address and port
10000. - For ROS#, verify the WebSocket URL and rosbridge port
9090. - Use the ROS computer's LAN IP when Unity runs on another machine.
- Ensure
ubm_interfacesis built and sourced before starting the endpoint or rosbridge. - Check firewall rules for the selected TCP port.
The vehicle does not move in ROS mode
- Select ROS odometry under Controls.
- Select the UBM FSAE topic preset or set
/ekf_pose/odommanually. - Run
ros2 topic echo /ekf_pose/odom --once. - Open the
F3HUD.ODOM: STALEindicates that Unity is not receiving fresh messages. - If messages arrive but tracking error grows, investigate pose application or coordinate conversion.
A custom message cannot be advertised
Build and source the matching ubm_interfaces package in the same ROS
environment as ROS-TCP-Endpoint or rosbridge, then restart the server.
The project opens with missing vehicle meshes
Install Git LFS, close Unity, run git lfs pull, and reopen the project.
Linux rendering artifacts
Restart the Unity Editor or standalone player with -force-vulkan. Graphics
API selection is not changed by the runtime graphics profile.