Here You will find all the information required to set up your own custom sandbox.
To run our software on you will need:
How to correctly place both Kinect sensor and the projector.
Sensor and projector placement without (left) and with (right) lens shift.
There are two things You'll have to bear in mind: the Augmented Reality Sandbox Software works best when the sensor is close to the light beam source and at the same time, it [sensor] should be placed approximately above the center of the sandpit. To meet both of these requirements we suggest using a lens shift projector. It may be slightly costlier, but will provide great quality of simulation and minimize errors.
Having the projector and kinect far apart may result in parallel shifts in the projection. Those are usually not significant, but still visible. We found that the best way to avoid this issue is to mount a lens shift projector close to the kinect and place them above the center of the sandpit. Such solution is only possible with this certain type of projector.
Bear in mind the Kinect sensor should be placed exactly above the center of the sandpit, facing downwards. See the overhead view image for clarification.
Drivers and packages necessary to run KinectSandbox and modules
Step by step tutorial.
For best results the application requires scrupulous calibration. The outcome of the process depends heavily on surrounding conditions, especially the lightning. It should be constant, evenly distributed and very slightly dimmed, soft daylight-like.
Make sure there are no flashing lights nearby, such as TV screens, or faulty light bulbs. Also cover the windows in case weather conditions (clouds, etc.) could affect lighting in the room.
The areas right next to the sandbox have to be nonreflective – avoid things such as oil-painted walls or tile floors. If necessary, cover them with white paper sheets.
To correctly gather calibration data, sensor needs smooth, flat and preferably white surfaces.
Our suggestion is to use cardboard boxes of three different sizes listed below. They should be arranged so that the edges of the same length contact each other and only the height varies. It’s best to not leave any gaps between boxes, however if they occur (eg. because of size of the sandpit), fill them with white paper sheets, so that the lowest surface is also plain white.
Box 1 – 20x20x20cm
Box 2 – 20x20x15cm
Box 3 – 20x20x10cm
Use this scheme to set up the boxes, starting from the edge of the sandpit:
Calibration automatic mode:
Calibration manual mode:
CalibrateEnsemble create new file (menu File > New, make sure Number of Projectors and Number of Cameras is 1) and save it to D:\ArSandbox_…\KinectSandbox_Data\StreamingAssets
It may happen that the calibration process doesn’t immediately give perfect results. This is because it uses and depends on physical objects and can be affected by their imperfections. Usually it’s a matter of few simple steps to reach the wanted level of accuracy.
The most common problem is projection being shifted compared to the actual shape of the sand. Reason behind this is the program misinterpreting or having wrong input data. First thing to do in such situation is to re-run the calibration software. If it doesn’t help, you should try repeating the actions described in the first chapter. As the lightning plays crucial role in the process, try changing it. Firstly, lower the projector’s brightness. Then, if necessary, brighten up the room. Avoid direct light sources and letting sunlight in.
Another problem that can occur is the projection going completely dark after launching the AR-Sandbox app with new calib.xml file. In such case go to D:\Calibration\procamcalibration\_RAT_ProCamCalibration, run the CalibrateEnsemble program as administrator, open the calib.xml file (menu File > Open…) from D:\ArSandbox_…\KinectSandbox_Data\StreamingAssets and check the number assigned to the display (menu Setup > Show Projector Server Connect Displays). You will see a number on the screen (should be either 0 or 1). Remember this number and then manually edit the calib.xml file (Open with Notepad). Find the line: <displayIndex>…</displayIndex> and write the display number in place of the dots.
One last thing to check is the number of lines in the newly prepared calib.xml file. It should be precisely 187. If it is not, it means the calibration process malfunctioned and has to be repeated. Pay close attention to all the steps.
Should you still have problems with your calibration, keep the old and new calib files and contact us at [email protected]
Quick overview of the most important settings inside the application.
There are some settings essential for the application to work correctly. These are the Sandbox Top and Bottom Levels. They specify the range of the Kinect sensor. See image for more details. Presented parameters are specified in the setting file. Their values are the distances in mm. The best way to achieve good results is to tape-measure the distances and then set the right values. It can be done either when the application is running (Z, X keys for Top Level; C, V for Bottom Level; values are shown on the screen), or in the setting file in application root folder (Settings.json in KinectSandbox_Data\StreamingAssets\Settings). If You choose to change the values with the running app, exit it (Alt + F4) to ensure they save. Also, we recommend that the values You enter have a margin of about 50-100 mm in both directions.In the graphic You can also see one parameter called WaterDropOffset. It is the distance above the Top Level of your sandbox, required to activate the waterflow. The higher the value, the higher You will have to lift your arm to make it rain. This parameter can only be reached via the setting file (Settings.json in KinectSandbox_Data\StreamingAssets\Settings). Value range 0-300.To directly control basic water options use computer keyboard or mouse. Press space or left-click mouse to reset water - it will sink into the ground. To disable waterflow press 'b' or right-click mouse. The same goes for enabling waterflow again. Notice: disabling waterflow does not reset current water, so You can have some in your drainage and then sculp it without unnecessarily pouring more.
Additional feature for expanded functionality
As the touchscreen module is now available as a software extension, you can implement it in your custom Sandbox. This allows for quick control over the application via easy to use interface.
For this feature to work, You will have to find a touchscreen that has exactly the same resolution as your projector. Then, when setting up both screens in Windows, make sure they are next to each other (horizontally, not vertically).
You will need Kinect 2 PC (it is Xbox ONE + PC adapter) You can buy refurbished one from the eBay.
Our system doesn't work with realsense. It is much much worse depth detector than Kinect 2. With Kinect 2 we get very accurate 3d model. We wish to use Azure Kinect in future.
We are going to release full video tutorial, as well as we want to update existing one. Operating buttons is very simple. You can use mouse or wireless mouse to be interface. Just get wires out form mouse buttons and connect your button. We also provide Arduino code for Arduino Leonardo/Micro pro control.
If you will make rigid frame for kinect + projector it will be stable. We have unit with wheels and tower that automatically goes up and there is no need to re-calibrate when moving unit. Calibration is very simple by the way - you need just white boxes.
Contact us, write about your project and let us think. At this moment price is very competitive.
We don't count. We wanted to have affordable price even for small organizations but if there is a science center with thousands of visitors a day they have to pay. It's just about being fair.