Calibration

Workflow Description

This article will go over the details of how to calibrate your cameras to match in 3D space as well as what to look out for if things go wrong.

Quick steps

Print a ChAruco board
Record different positions
Using that recording calibrate with Stereo Calibration.

Detailed Steps

Make sure you have the newest build downloaded from here: https://spatialscan3d.com/downloads/
Prepare the calibration board
1. Print a ChAruco board.
  1. Open Studio and navigate to the Calibration Tab.
  2. Expand the Stereo Calibration function.
  3. Choose one of the three board presets.
  4. Click the PDF icon to the right of your choice.
    This will open a folder that and select the preset that you chose. Print this pdf on a matching size paper (or another material that is able to avoid light glare).
    
    Note: When printing, check to make sure that the pdf is not stretched or altered. It’s also best if you can print it on the same size of paper as the chosen preset.
2. Measure ChAruco parameters
  1. Measure the length in millimeters the distances of bigger and smaller black square edge lengths.
  2. If they match the numbers at the edge of your printed preset, then you don’t need to do anything else, but if they are different you can either print a new board, this time ensuring that the print is not stretched or shrunken down as mentioned earlier, or you can use the board that you printed, but then you’ll need to write down you actual measurements because they will be needed later.
3. Attach the ChAruco board to something flat and stiff
  1. During calibration the board must not bend such as simple paper tends to do, so attach the printed board to something hard, such as cardboard.Note: If you are using tape to attach the board to something else – use non-transparent tape to avoid light glares during recording.
Finalize your camera setup
1. When in the end you’ll calibrate your cameras, that calibration will only work as long as the cameras are not moved.
2. Place your cameras in the range of 0.5 meters -> 1.5 meters away from the center of your stage.
3. Set your cameras up in such a way as to provide as much overlapping area between the pairs of cameras that you will be calibrating (calibration pairs will be explained later in this article)Example: If you are working with 4 cameras, you should set them up at 90 degree angles, at about the same height, ~1 meter away from the center, thus opposite cameras will be ~2 meters away from each other.Note: If your cameras are further away from the center, the quality of the calibration and the pointcloud will likely suffer, and in extreme cases you won’t be able to calibrate altogether.
4. If you have synchronization cables, connect them to your cameras.
5. Attach your cables to something so that they wouldn’t be accidentally bumped into and moved.
Choose your recording settings
1. Navigate to the Record tab.
2. Set the resolution to FHD, unless your cameras are further away from the center than 1.3 meters, in which case use QHD resolution.
3. Set the framerate to 15fps. For Calibration this is enough and will make the process faster.
4. Disable the Show 2D Depth Masks toggle within the Cameras feature. This is because the stereo calibration process uses the whole color image, and so it’s good to see your true boundaries of what the cameras can see.
5. Adjust Color Controls
  1. Turn your cameras on and enable their 2D toggles.
  2. Exposure time should be as low as possible to minimize motion blur.Note: In case the colors become too dark, try to increase Gain, even to the maximum if needed, and if even that is still not enough brightness, bring the Brightness slider up, but try not to increase it too much as it creates washed out colors
  3. Adjust the White Balance slider until the colors in the 2D images match the colors in reality.
6. If you are using multiple computers to record with, check out how to use the Network feature.
Record you calibration video
1. Make sure that all the cameras that you wish to calibrate are turned on
  
  Note: 2D display can be turned off, for it does not affect the recording, thus if you are experience frame loss, consider turning 2D off for some of the cameras.
2. Click Start Recording.
3. Take the ChAruco board into your hands.
4. Move to the center of your scene.
5. Show the board to your camera pairs
  1. Decide which cameras are gonna need to be paired
    1. All of your cameras need to have a continuous path towards all the other cameras. The best way to do this is to create a daisy chain.Example: If you have 4 cameras standing in a circle, all labeled [1], [2], [3], [4], then to link them all together the chain would look like this: [1]->[2]->[3]->[4], where a pair of cameras would be both cameras on either side of each arrow, meaning this chain would have 3 pairs: Pair 1: [1] and [2], Pair 2: [2] and [3], Pair 3: [3] and [4].Note: Our calibration is not exactly a daisy chain, because you can have multiple cameras being paired with a single other camera, so if we have our previous example, but add a fifth camera [5] that is perhaps above camera [2], then you could have a chain like this: [1]->[2]->[3]->[4] + [5]->[2].
  2. To each camera pair, show different board positions/angles
    1. Hold the ChAruco board in your hands.
    2. Step in front of the two cameras that you are calibrating and turn the board towards the center between them.
    3. Show at least 5 positions to these two cameras (showing more should give better results, but don’t overdo it)
      1. A position in this context is a different position and orientation of the ChAruco board, where both of the cameras are able to fully see the board.
      2. When you move the board to a new position, it is import to then keep the board as still as possible so that there would be as little motion as possible. This is made easier by holding the board in that new position for at least one second, but for your first attempts I suggest holding it for two seconds in each position.
      3. It’s important to show at least one position which is as close as possible to the cameras (keep in mind that Both cameras need to see the board fully when you hold it in a position). Try to show the board from higher up, middle height and lower down. Try moving to the left and right, as well as backwards and forwards.Note: When moving backwards, try to not go further than 2 meters away from the cameras.
6. Go back to your computer and Stop Recording.
  
  Note: This will then create your calibration video as an .eve file within your chosen directory (this can be chosen before recording by clicking the button bellow Start Recording).
7. Give a name to the new recording, and save it.
  Note: This will then create your calibration video as an .eve file within your chosen directory (this can be chosen before recording by clicking the button bellow Start Recording).
Open your calibration video
1. Head to the Input tab and within the Open Eve Files function click the Open Files button.
2. In the opened file dialog, navigate and choose your recorded .eve file and then wait for it to load.
Use Stereo Calibration
1. Go to the Calibration tab and open the Stereo Calibration feature.
2. Select the ChAruco board preset that you printed and used for recording this calibration volumetric video.
  1. If your ChAruco board lengths are different than the preset then do these steps:
    1. Open the Charuo Settings
    2. Change the first two sliders to match your own measured lengths (in millimeters).
3. Click the Calibrate button.
4. Wait for calibration process to finish.
5. Check the calibration quality
  1. The simplest way to check is through the pointclouds in the Live Preview of the 3D Window. If the calibration is bad you’ll be able to easily see big offsets in your pointcloud.
  2. If the calibration at least somewhat succeeded, to check it’s quality you should:
    1. Expand the Calibration Sessions
    2. Expand the latest session (usually Session 1)
    3. Here, for each camera pair that was calibrated you’ll see a line which has a number in the middle that tells you how well it was calibrated. The closer this number is to 100 the better the calibration. Getting values above 70 is what you should aim for.
6. If the calibration quality is at least somewhat decent then follow these steps:
  1. Make sure that the synchronization of the recording is correct
    1. Use the playback tick with your mouse and drag it to a part in the volumetric video where there was fast movement happening.
    2. By moving the playback back and forth in that area, try to visually detect if any of the cameras are lagging behind
    3. If any of your cameras are lagging behind, you need to:
      1. Synchronize the recording:
        Drag the camera/cameras in the timeline that are lagging behind until they match up with the other cameras.
        You can drag a single camera to change it’s position in time by clicking and holding on any of that camera’s frame bits in the timeline, and then moving it left or right. While dragging a camera you should look at the 3D pointcloud and see when the camera is at the correct position in time.
      2. Expand the Calibration Manager feature at the top of the Calibration tab and click the Reset button.
        This will remove the previously done calibration, because it was done using wrongly synchronized data, which you now corrected.
      3. Ho back the the Stereo Calibration feature and click Calibrate once more.
  2. Do the Floor calibration
    1. How to use
  3. Try to improve the current calibration by adjusting the Stereo Calibration sliders.
    1. Go back to the Stereo Calibration feature, and move the sliders.
      1. Max Overlap Threshold should give better results the lower it is.
      2. Min Angle Threshold should give better results the higher it is.
      3. Max Pairs To Use should be kept as is, but you can experiment to see if different values give you better results.
      4. Min Motion Frames will give better results the lower the number is, if during recording you were able to hold your board very still. Otherwise higher values are recommended.
    2. Click the Calibrate Again button and wait for the calibration to finish.
    3. Check the Calibration session quality numbers.
      1. If the values increased, try to move the sliders that you changed even further.
      2. If the values decreased, move the sliders in the opposite direction than you did.
    4. Repeat this process until you reach an equilibrium, where moving any slider would result in a worse result.
  4. Save your calibration to a .clb file.
    1. Navigate to the Calibration Manager feature.
    2. Click the Save button.
    3. Choose the folder where to save your calibration file in the opened dialog window.
  5. You are now done. You can now continue to record volumetric videos using your current camera setup and whenever you open your newly recorded .eve files just make sure to go to the Calibration Manager and Load your calibration .clb file.
Troubleshooting / Stereo Calibration Failed
1. Sometimes the calibration will fail and so here’s what you can do in that case.
2. First let’s figure out what failure is. You can see that a calibration failed the same way you can see if it succeeded:
  1. Visually in the 3D Window.
  2. By numbers in the Calibration Session.
  3. Within the Calibration Session, there won’t even be any Sessions created.
3. What does it mean if there are no Sessions created?
  Most likely this means that Stereo Calibration algorithm went through the whole recording and after all the filtering was left with 0 common frames between all cameras. This can happen if:
  1. The filtering is too strict, in which case you should try any or all of these:
    1. Increase the Max Overlap Threshold
    2. Decrease the Min Angle Threshold
    3. Increase the Max Pairs To Use
    4. Increase the Min Motion Frames
    5. After making any of these changes, Reset your calibration in the Calibration Manager and click Calibrate in the Stereo Calibration feature.
  2. Either the cameras were missing frames and thus managed to miss all the frames where you were showing the ChAruco board to the cameras, or it can also be that for one reason or another the cameras simply did not see the board, or at least did not see it fully, or well enough. To fix this you should:
    1. Check to make sure that no frames are being missed during recording.
      1. If they are being missed:
        try turn off 2D display
        
        Try connecting your cameras to a different usb port.
    2. Ensure that the cameras see the board properly
      1. Ensure that you are not standing too far away from the cameras.
        It’s best if you’re within 2 meters of both cameras.
      2. Make sure that light glare is not causing issues on your board.
      3. Make sure that lighting is not too bright or too dim, there needs to be a good contrast between the white and black colors of the ChAruco board.
      4. Whenever possible, try to reference the 2D display of the cameras that you’re trying to calibrate during the calibration recording, to ensure that you are not holding the board outside the camera’s boundaries.
      5. Make sure that you’re not holding the ChAruco board at very extreme angles from the perspective of the cameras, because while showing the board at an angle is good thing, having that angle be very big, makes the cameras unable to detect the board.
4. What to do if only some of your cameras were calibrated and some are missing from the Calibration Session?
  This happens for the same reason as explained above for when no session was created, except that it was able to find some common frames for some of your cameras but not for all. Thus do the same steps as detailed above.
5. What to do if the calibration quality is not very good?
  So you managed to calibrate all of your cameras, but the quality in the Calibration Session is bad (the number is lower than 70)? Here’s what you can try that might make your calibration better:
  1. If possible, try to move your cameras closer to the center, or closer to each other and re-do the whole calibration process
  2. Try to improve the current calibration by making another calibration recording:
    1. After getting your first calibration results, Save them to a .clb file using the Calibration Manager.
    2. Record a new Calibration video, same way as you did you first video, but you can choose to only show the ChAruco board to those pairs of cameras that you wish to improve.
    3. Close any previously opened .eve files in the Input tab, and open the newly recorded volumetric video.
    4. Load the previously saved calibration .clb file.
    5. Ensure that the new video is synchronized correctly. (Check 7.6.1. for instructions about it)
    6. In Stereo Calibration click the Calibrate button. This should then create a new Calibration Session, and as long as the program is able to get a better calibration for any of your camera pairs, it will then use those better calibration results from the new session.
  3. Record a new calibration video from scratch and do the whole process again, this time trying out different ChAruco board positions, perhaps holding it a bit more still when in positions.
  4. Try to adjust the calibration manually, using the Manual Calibration feature
    1. How To Use
  5. If none of these options work, try to contact us with a clear description as to what was done and what went wrong, and we’ll try to find a way to fix it together. You can find our contacts here: https://ef-eve.com/contact-us/