PointCloud v3 Changelog

v3.16

• first official non-Beta release
  • installer no longer expires
  • your maintenance year starts on the v3.16 release date, also for licenses bought during the beta
• added Help menu item “Kinect v2 Keeps Disconnecting (Windows 11 Fix)” linking to a troubleshooting page
• Kinect v2 sensors now detect that disconnect/reconnect loop and explain it in the message log, pointing at the help page above
• added Help menu item “USB Bandwidth Diagnostics” showing what every USB device on the machine costs and where a multi-sensor setup runs out of room
• added a “USB MB/s” column to the sensor table, with link speed and reserved bandwidth in its tooltip
• USB problems are now reported in the message log, once each, and again if they come back
• fixed a crash in the 3D viewport while playing back or scrubbing clips from the timeline (a clip could hand the viewport a new frame while the graphics driver was still reading the previous one)
• the 3D viewport no longer briefly hitches when a clip delivers a frame in the middle of a redraw

BETA v3.15

• memory usage and thread safety optimization in BPC reader
• fixed a crash with offline alignment due to multi-threaded interplay with new data fusion solvers in some cases
• fixed issue with offline alignment export not saving sensor names to file
• fixed OpenGL drawing issue when a dock is torn of the main window while it’s running
• fixed network sensor recording not working (clients didn’t record a BPC and the live stream froze when starting a recording)
  • recording start/stop signals to network clients are now sent on the main thread (cross-thread socket sends were silently failing)
• marker-based sensor alignment now explains why it failed, in plain language, in the log (both live and offline from clips)
  • e.g. a sensor never saw the marker, two sensors never saw it at the same moment, marker too small/tilted/fast, low quality, or different markers per sensor
  • each reason comes with a suggestion on what to change, plus a metrics line for support
  • reduced the per-frame logging noise during live marker alignment so the diagnosis stands out

BETA v3.14

• various small fixes and improvements after code analysis
• added new meshing algorithms
  • besides “Quick & Dirty” there’s now also “TSDF Fusion” and “Poisson Fusion”
  • these come with new texturing algorithms: “Winner Takes All”, “Multi-View Bake (Fast)”, and “Multi-View Bake”
  • mesh fusion parameters are now saved/loaded with settings files and persisted across sessions

BETA v3.13

• improved alignment quality
  • marker-based alignment (for both live-sensors as well as offline from clips on the timeline)
  • as well as body-based alignment (offline from clips on the timeline)
  • improvements use newer more optimized math for the solver
  • better outlier rejection and quality selection
  • better temporal alignment guards
• fixed rare audio related crash

BETA v3.12

• made gathering some system info like GPU info and usage a bit more robust
• fixed issue where video recorder worked the first time but could fail on subsequent recordings
• fixed visual anomaly where video streams of some network sensors could appear horizontally mirrored (note that this did not affect the pointcloud or skeleton data)

BETA v3.11

• fixed issue with missing file in installer causing (file) alignment using bodies to fail
• fixed race condition in (file) alignment using bodies
• recorders are now started & stopped in parallel threads to avoid potential acummulating delays across sensors (like relatively slow disk access for example)

BETA v3.10

• added transform gizmo to translate/rotate sensors
  • select sensor (or clip) in the bottom tab
  • CTRL + T in 3D viewport for Translation mode
  • CTRL + R in 3D viewport for Rotation mode
  • CTRL + E in 3D viewport to toggle between Local and Global coordinates
  • CTRL + Z for undo and CTRL + SHIFT + Z (or CTRL + Y) for redo work
• added an option to the GUI to explicitly set the Marker Style from Traditional to Modern or relying on Auto detection
  • we previously relied on auto detection but this may sometimes fail and cause alignment using markers to fail
• overhauled how external libraries and drivers are loaded
  • sensor dlls and their dependencies are now isolated into their own folders
  • opens up some future expandability
  • reduced installer size
• improved stability for Microsoft Azure Kinect and Orbbec Femto sensors
• fixed potential race condition when loading BPC frame and copying to GPU for display
• fixed issue with Orbbec Astra sensors not working on some systems
• fixed some small vertical stretching anomalies in the GUI
• optimizations in the timeline code
• updated expired discord link in help menu
• updated to latest ZED SDK v5.07 (with CUDA12 & TensorRT 10)
• updated expired discord link in help menu

BETA v3.09

• added option to crash feedback dialog to opt for never providing feedback
• added “-no_crash_feedback” commandline option to suppress crash feedback dialog at all times
• added additional checks to prevent rare crash when USB communication may have produced invalid data buffer
• improved uninitialize method for ZED sensors
• improved uninitialize method for Orbbec Astra sensors
• improved, streamlined, styled and updated installer

BETA v3.08

• switched to a new crash detection & logging system (crashpad)
  • this should be able to detect a wider range of crashes and provide better debugging information
• optimized internal pixel/point loops for Kinect v1 & v2 sensors a bit
• cleaned up some GUI layouts
• improved sensor initialization code for console version
• improved realsense polling for frames to be non-blocking
• when loading an alignment file the code now first checks if it’s a perfect match and only displays the mapping GUI if there is a mismatch
• added -load_alignment_file commandline option
  • attempts to load alignment file from the given path after sensors have connected
  • add something like load_alignment_file “C:/folder/file.txt” as a commandline argument
• fixed issue with new version being available message would still appear even if user disabled it
• fixed a multi-threaded deadlock that could happen when connecting 2 or more client machines
• fixed a small race condition in one of the math class operations
• fixed potential lock/stall when connecting multiple clients at once
• fixed issue with shortcut “start in” not pointing to the installation folder
• fixed internal memory management for timers
• fixed potential race condition in Azure Kinect / Orbbec Femto class when setting up internal vectors buffer at initialization
• fixed potential deadlock in sensor initialization
• added hardening and more logging for initialization of Body Tracker for Azure Kinect / Orbbec Femto sensors
• added some additional logging surrounding network client discovery and network sensor initialization
• implemented a more robust way to query GPU info to avoid rare case of crash especially in console at startup of deep tracker
• improved FPS calculation to avoid memory copy and fixed very rare potential race condition
• additional hardening in network and sensor classes to prevent race condition expecially in multi-client scenario
• changed the memory managment in the network server class and how it relates clients to ids to be more rubust and thread friendly
• increased number of internal data buffers to be more on the safe side
• suppress network buffer warning during startup as it’s not relevant
• implemented a more robust way to query GPU info to avoid rare case of crash especially in console
• updated to latest Intel RealSense SDK v2.56.5
• updated to latest ZED SDK v5.05 with CUDA12 & TensorRT 10

BETA v3.07

• implemented a new data buffer pool that should be simpler, faster, more robust and more thread-safe in edge cases
  • for both the sensor class and networking classes
• implemented a better buffer overflow and buffer flushing mechanism in the network classes to recover from corrupted data and other edge cases
• improved network buffer hardening against corrupted packets
• improved time sync calculation in case there are multiple queued up packets to be handled, especially with multiple client scenarios
• fixed a potential edge case in how console handles sensor IDs in case there are multiple
• fixed issue with one call in the network server class not respecting the use_selected_adapter_for_UDP switch
• fixed minor cleanup leak on server side upon closed connection of a client
• fixed inclusion of old realsense dll file
• various small improvements after static code analysis

BETA v3.06

• fixed link to online shop
• fixed potential crash with network sensors while receiving sensor stream info when things hadn’t fully initialize yet
• fixed potential edge case for a race condition in the network server class
• fixed issue with video stream not being displayed in 2D widget for some network sensors
• optimized network sensor data compression and decompression speed
• optimized several of the pixel loops for Azure Kinect / Orbbec Femto sensors
• added extra checks for sensor validity to prevent potential crash
• added some additional validation checks in the marker alignment to try and catch non valid data to prevent potential crash
• added some extra timeout handling for Orbbec Femto & Azure Kinect sensors
• added some small checks in the console string drawing routine

BETA v3.05

• fixed issue with preferred network adapter state of “Auto” not being correctly saved it would always revert to saving the actual ip
• fixed issue with network sensor id logic especially when using multiple machines and/or the local machine had a sensor attached as well
• added some additional memory safeguards for Azure Kinect / Femto Bolt
• UDP socket is now more expclicitly bound to selected network hostname at startup instead of broadcasting to all
• improved compiler linking optimizations

BETA v3.04

• improved how unique_ids are generated when a network client connects
• implemented a more sophisticated method to synchronize timestamps between networked machines
• improved memory handling for Azure Kinect & Femto Bolt sensors to improve robustness
• added buttons to save/load camera pose presets for the 3D viewport
• added toggle to enable/disable drawing of status text in viewport
• improved crash report handling code
• fixed issue with manually selected network adapter not always being used on app startup
• fixed potential memory leak for Azure Kinect and Orbbec Femto sensors
• fixed issue with console/headless not resetting some timestamp sync defaults on disconnect
  • this could result in falling back to a lower accurate timestamp synchronization method after disconnect/reconnect
• updated to Intel RealSense v2.55.1
• updated to ZED 5.x drivers/SDK
• updated to new build environment
• installer and app are now signed by a new EV code certificate

BETA v3.03

• added a few runtime checks against uninitialized sensors that could cause potential but rare crash
• updated various internal libraries to latest versions

BETA v3.02

• on rare cases where user has no GPU drivers supported and OpenGL calls fail we now try to write an error to the message log
• changed internal Orbbec log settings, as it could crash attempting to create a file in “Program Files” folder without having permissions
• reverted back to old way of checking RTX support (by looking at the GPU model string)
  • as new method (by looking at windows device specs/stats) sometimes reported RTX not being supported on GPUs that did support it on some client machines (more relevant for Body v3 and not so much for PointCloud v3)

BETA v3.01

• fixed issue where live sensor transform setting was lost when swapping to File tab and back to Live
• fixed issue with sensor DLLs being compiled against NVIDIA CUDA SDK and not working on machines without an NVIDIA GPU
• NVIDIA GPU/CUDA/RTX support is now checked based on what Windows reports on the system’s GPU instead of relying on NVIDIA libraries
  • used to detect which deep learning models are supported
  • may be a bit less reliable but it should work on all systems, including those without an NVIDIA GPU
• disable Azure Kinect firmware check (since Microsoft isn’t updating it anymore anyway)

BETA v3.00

• fixed issue where app could sometimes lock up when adding clip to timeline due to multi threading mutex deadlock
• changed version numbering scheme to start counting from Beta 3.00

BETA v0.973

• fixed some issues with Azure Kinect sensor startup in some cases
• fixed issue with binary PLY with vertex colors
• fixed race condition crash in timeline code
• added a compiler flag to better support printing Unicode characters to the console window
• updated various internal libraries
• updated internal libraries to latest versions
• updated to OrbbecSDK v1.10.11
• updated to FBX 2020.3.1
• updated to FFmpeg 5.0.1
• updated to Visual Studio 2022 with all the latest compiler optimizations

BETA v0.972

• fixed issue with batch processor not being opened when button was clicked
• fixed issue with camera transform in FBX export not lining up properly
• fixed issue with camera transform in FBX export not respecting “Scale Multiplier”
• fixed issue with sensor state not always being saved
• when headless app version does not match GUI it now displays a message box
• keyboard are now defined globally so it doesn’t matter which widget is currently active
• optimizations based on static code analysis
• added CTRL + F1 keyboard shortcut to show/hide left dockwidget (drawing / selected sensor settings)
• added CTRL + F2 keyboard shortcut to show/hide right dockwidget (settings)
• added CTRL + F3 keyboard shortcut to show/hide right dockwidget (recording)
• added CTRL + F4 keyboard shortcut to show/hide bottom dockwidget (sensors table / message log)
• updated to latest OrbbecSDK v1.9.5

BETA v0.971

• fixed regression that caused a crash when starting a recording with Azure Kinect sensor
• optimizations based on static code analysis

BETA v0.970

• double clicking a clip on the timeline now selects it, double clicking a selected clip deselects it
• compacted console drawing a bit
• added option to the top Settings menu to reset settings
• moved save/load settings from File menu to Settings menu
• fixed issue with some settings not being saved/loaded to/from file (while Windows registry did store/load them)
• extended beta to 1st of August

BETA v0.969

• improved speed of reading frames from BPC file recorded with Azure Kinect by more efficiently multi-threading
• improved speed of mesh normal calculations by more efficiently multi-threading
• improved speed of EXR exporter by more efficiently multi-threading
• fixed alpha of EXR exporter
  • pixels with non-valid depth are now black in alpha, pixels with valid depth are white
• moved “Show Advanced Options” from top menu to Settings tab so it’s more visible for users
• fixed issue causing a crash on sensor discovery on some (Win11) systems, depending on other installed programs
  • added more logging when discovering sensors to help identify and isolate such crashes in the future
• fixed issue of GUI would sometimes not re-enabling until timeout when no (valid) sensors were selected upon “connect sensors”
• fixed issue where sensors disabled in “advanced sensor settings” remained disabled even when not using advanced settings
• fixed issue with Orbbec drivers not always being detected
• fixed issue where “Custom Marker Length” on Live tab was hidden when “Settings > Show Advanced Options” was disabled
• fixed texture size output to message log being incorrect (export was correct)
• fixed issue with “Advanced Sensor Settings” dialog crashing on subsequent uses
• fixed issue where sensors that a user disabled in the “Advanced Sensor Settings” dialog could still show up in the sensor table
• some efficiency optimizations for Kinect v1 (XBox 360) sensors
• ffmpeg no longer pollutes console with lots of debug messages during video output
• updated to latest compiler
• updated internal libraries to latest versions
• updated to OpenCV 4.8
• updated to latest Orbbec SDK v1.8.1
• updated to latest Intel RealSense SDK v2.54.2

BETA v0.968

• fixed issue with dll file not loading on release version

BETA v0.967

• fixed potential race condition crash that could happen for some image exporters
• fixed issue with Trial version not finding Orbbec Astra+, Gemini and Femto sensors due to missing file

BETA v0.966

• added support for Orbbec Astra+, Gemini and Femto sensors (using new SDK)
  • Orbbec Astra and Astra Pro sensors (using older SDK) are still supported as well
• improved marker type detection
• updated to OpenCV 4.8
• updated libraries to latest versions
• extended installer to 1st of February

BETA v0.965

• fixed issue with network sensor not setting texture stream correctly causing pointcloud to be rejected in GUI
• “Display Fused Data” is now automatically enabled during export (and set to it’s original state afterwards)
• updated internal libraries to latest versions
• updated to latest Intel RealSense SDK v2.54.1

BETA v0.964

• fixed occasional issue with Alembic exporter crash on premature file close when frame export hadn’t finished yet
• exporter now logs amount of points, polygons and texture size per frame to help debugging
• various small code optimizations based on static code analysis
• some efficiency optimizations for Kinect v1 (XBox 360) sensors
• ffmpeg no longer pollutes console with lots of debug messages during video output

BETA v0.963

• added option to switch between “Advanced Sensor Settings” and regular setting selections
  • when advanced is enabled you can specify per sensor to
    • use it or not
    • use color or infrared (depending on sensor support)
    • which resolution to use (depending on sensor support)
• fixed some issues with using advanced sensor settings in some cases
• fixed some issues with UI widgets when using advanced sensor settings in some cases
• fixed issue with crash in BPC reader when skiping points section on some file variants
  • this could manifest itself when doing an offline alignment using markers and a Kinect v2 file for example
• updated internal libraries to latest versions
• updated to latest compiler
• extended installer to 1st of September

BETA v0.962

• updated internal libraries to latest versions
• extended installer to 1st of August

BETA v0.961

• minor efficienty improvements for internal processing of several sensors
• changed a string conversion routine in record triggering class to be more robust for certain unicode characters
• updated to latest Stereolabs ZED SDK v3.8.0
• updated to latest Intel RealSense SDK v2.53.1
• extended installer to 1st of April

BETA v0.96

• fixed issue with Azure Kinect sensors running on headless client didn’t send data to GUI
• fixed issue with timetamp being displayed as 0.0 if sensor was selected while running infrared instead of color
  • this also fixes a warning in the message log about timestamps not matching in multi-sensor setups
  • this could potentially fix some issues with alignment too
• added mutex to headless version to avoid (rare) multi-threaded race condition
• when clocks of network clients and GUI are synchronized this is now logged in the GUI
• extended installer to 1st of January

BETA v0.95

• for Azure Kinect multi-sensor sync is now automatically disabled when using Infrared stream (since drivers only support this for color)
• fixed memory leak (eventually causing a crash) for Kinect v2 when using infrared in headless mode
• added color whitebalance controls (for sensors that support it)

BETA v0.94

• added “Advanced Sensor Settings” window
  • finds and list all sensor types found on your machine
  • allows to enable/disable usage of each sensor
  • allows switching between color/infrared when supported/applicable
  • allows to switch video/depth resolution when supported/applicable
• added critical warning message when USB2 cable is detected on RealSense sensor
• added Desktop, My Documents and Home shortcut locations to sidebar of file/folder dialog windows
• added option (from File menu at the top) to save/load program settings to/from .ini file
  • settings are also still automatically saved/loaded to Windows registry on application close/start
• removed option to switch resolution while sensor is running (as this didn’t work well on all sensor types)
• removed “Edit Sensor Blocklist” since this functionality is now handled by the “Advanced Sensor Selection” window
• removed “Force using Color for video” from top menu since this functionality is now handled by the “Advanced Sensor Selection” window
• fixed a crash that could happen on sensor uninitialize
• preferred network adapter window will now open up with previous setting instead of always reverting to Auto
• updated to latest Intel RealSense SDK v2.51.1
• extended installer to 1st of November

BETA v0.93

• at application start we now inform Windows OS to prefer not to go to sleep (released at application stop)
• small additions and corrections in documentation
• when running an Intel RealSense sensor on USB2 we now print a warning in the message log
• override default depth units for Intel RealSense D4xx sensors to increase depth fidelity
• fixed issue with some of the lens calibration data of Intel RealSense sensors not being read correctly from drivers/SDK
• fixed issue with marker detection on some Intel RealSense sensors
• updated to latest Stereolabs ZED SDK v3.7.6

BETA v0.92

• fixed link to official Microsoft site for Kinect v1/v2 drivers
• added link to Discord in the Help menu
• updated to FBX 2020.3.1
• updated to Aruco 3.1.15
• updated to FFmpeg 5.0.1
• updated to Visual Studio 2022 with all the latest compiler optimizations
• extended installer to 1st of September

BETA v0.91

• fixed issue with Low/Hi resolution mode for Azure Kinect clips
• StereoLabs ZED2i sensor should now be recognized
• refined driver detection
• updated internal libraries to latest versions
• updated to latest Stereolabs ZED SDK v3.7.4
• updated to latest Azure Kinect v1.1.2 SDK

BETA v0.90

• improved visual quality of network sensor data
  • especially noticable on lower quality settings
  • more aggressive edge filtering on 3D pointcloud and color video streams
  • has no effect to tracking/recording quality (since that is all done on the remote machine on uncompressed data)
• extended installer to 1st of July

BETA v0.89

• fixed issue with marker detection points
• updated to latest Azure Kinect v1.1.1 SDK

BETA v0.88

• added option to enable hardware acceleration for video exporter for faster video encoding
  • attempts to use NVIDIA, AMD and INTEL acceleration and automatically falls back to CPU mode when not available
• fixed a bug where MJPEG video codec did not work due to unsupported video header setting
• H264/H265 now explicitly use “main profile” making files potentially more compatible with various players
• updated to latest Stereolabs ZED SDK v3.7.0
• extended beta to 1st of May

BETA v0.87

• added option to switch 3D view to fullscreen by double clicking on it
  • can be disabled by toggle from the Settings menu at the top of the app
• updated internal libraries

BETA v0.86

• fixed issue with JPEG compression/decompression that could cause a crash on recording and possible at other points on some machines
• updated to latest Intel RealSense v2.50.0

BETA v0.851

• fixed issue with audio files not recording
• fixed issue where color resolution would be hidden (due to overlapping code from webcam integration for Body v3)
• potential fix for issue with exporter crash on fast machines and many outputs (extremely hard to reproduce & test)
• implemented a different strategy to prevent race condition in 3D view

BETA v0.85

• added keyboard shortcuts for working with timeline
  • Home: rewind to first frame
  • Space: play/pause
  • Left arrow: previous frame
  • Right arrow: next frame
• Added “Save to File” button for offline alignment
• alignment files are now saved with full transform matrix for added precision when loading them
• fixed issue with Azure Kinect Hirez setting not always loading correctly from timeline file
• fixed issue where Intel RealSense sensors could invoke deep body tracker even when not requested on a frame
• fixed incorrect warning that “timestamps vary too much” when sensors took a while to initialize
• start recording will now switch lower tab to “Sensors Table”
• added some extra info to the documentation regarding USB bandwidth
• updated to latest Stereolabs ZED SDK v3.6.1
• extended beta to 1st of January

BETA v0.84

• timeline clips now have their hold first/last frame toggles enabled by default
• fixed issue where pointclouds would keep disappearing after switching to live tab and then back to file tab
• fixed issue where time range for offline alignment didn’t automatically change when timeline bounds changed
• updated to latest Stereolabs ZED SDK v3.5.5
• extended beta to 1st of November

BETA v0.83

• fixed memory leak in BPC reader when reading high resolution Azure Kinect frames
• various small bug fixes and optimizations
• updated internal libraries to latest versions
• updated to latest Intel RealSense v2.49.0
• extended beta to 1st of October

BETA v0.82

• fixed issue with mesh decimation setting not loading correctly
• improvements regarding GUI update when timeline clips change
• implemented a new internal threading model for handling sensor data
  • fixes issue of AMD processors reporting 100% utilization even when many CPU cycles were simply idling
  • better compatibility for Intel & AMD CPUs
  • slightly more efficient in some cases
  • also implemented for network data compression
• updated to latest Stereolabs ZED SDK v3.5.2

BETA v0.81

• small internal change for Azure Kinect on how valid frames are checked and released to prevent chatty SDK messages to appear when computer can’t keep up processing fast enough
• updated CPU body detection deep learning model and platform to latest version
• updated some libraries to latest version
• updated to latest Intel RealSense v2.48.0
• extended beta to 1st of September

BETA v0.80

• fixed issue with OpenEXR output being mirrored compared to 2D/3D views in Brekel app
• fixed issue with OpenEXR Azure Kinect color data being mirrored compared to other streams
• OpenEXR data is now filtered to prevent NaN numbers (just in case)
• OpenEXR depth and position channels are now embedded in layers to adhere to Nuke’s standard
  • depth.z
  • pos.posX
  • pos.posY
  • pos.posZ
• fixed issue with Kinect v2 sensor occasionally crashing on unitialize
• fixed issue with printing sensor calibration values
• network server thread now caches incoming UVs where possible

BETA v0.79

• fixed issue where “Cancel Export” button was disabled in GUI during an export
• implemented new OpenEXR texture output with options for saving several channels
  • RGB - saves the color information to the OpenEXR “RGB” channels (color conformed to depth)
  • Alpha - saves data to the OpenEXR “Alpha” channel, white for pixels that had a valid depth reading from the sensor, black for invalid pixels
  • Raw Depth - saves the raw depth information (the distance to the sensor) to the OpenEXR “Depth” channel
  • XYZ - saves the global XYZ positions (with sensor transforms baked in) to the OpenEXR “XYZ” channels
  • some additional sensor related data (like sensor type, transform and calibration data) is written as custom data in the OpenEXR header

BETA v0.78

• consolidated post-processing and output settings (on export settings tab) into a single tool box (instead of two) for better clarity
• reworked the offline alignment GUI a little bit
• reworked the internal aligner class
  • now can do a second pass using an all new algorithm refining all sensors collectively
• implemented a new body-based calibration option that no longer needs a printed marker, just a person
  • can use single pass and two pass alignment
  • can do an extra refinement method using point data from nearest sensors for tighter fit
  • currently only available for offline methods (live/realtime version coming in a next version)
• made camera panning speed proportional to camera distance so when the camera is far away it visually moves a similar distance when panning
• added a depth multiplier for PNG depth output
• fixed issue with OpenEXR depth export (due to bugfix in underlying image handling library no longer needing workaround)
• fixed issue where OpenEXR/PNG depth would only export a single clip
• fixed issue where only selecting PNG depth textures would result in a “no output formats selected” error
• fixed issue where headless/console version would crash when checking GPU support (since OpenGL context was not valid)
• fixed issue with internally selected sensor not being reset on sensor disconnect
• fixed issue where offline marker detector could crash on rare occasions
• updated internal libraries
• extended beta to 1st of August

BETA v0.77

• added chapters to the documentation with some sensor specific information
• updated to latest Intel RealSense v2.45.0
• extended beta to 1st of July

BETA v0.76

• updated to latest Stereolabs ZED SDK v3.5.0
• fixed bug that produced noise on Kinect v1 color stream
• switched to different debayering method for Kinect v1 producing faster and slightly better results
• network pointcloud processing will now drop frames if there are more than one in the queue
  • which can potentially happen when a machine cannot keep up with decompressing frames
  • this will only affect visuals in the 3D viewport and does not affect pointcloud recording
• if the software detects timestamp mismatches the table will reflect this with colors
• machine and sensor info is now logged to help debugging- added additional check for invalid image buffer size in memory management
• added additional check for invalid image buffer size in memory management
• timestamps are now periodically logged to help debugging
• updated several internal libraries to latest versions
• updated to latest Stereolabs ZED SDK v3.5.0

BETA v0.75

• fixed memory leak in BPC recorder for Azure Kinect sensors
• console now resets clock offset when server disconnects
• timestamp synchronization algorithm for clients will now automatically fallback to using TCP packets when UDP failed somehow
• improved internal body time interpolation algorithm
• added “Select Clip” and “Unselect Clip” option to the context menu when right clicking on a clip
• updated to latest Orbbec SDK
• extended beta to 1st of June

BETA v0.74

• added toggle to sensors table to toggle visibility per sensor (they will still be recorded)
• added a warning message to the log on older CPU’s that do not support AVX2 or AVX instructions
  • this may prevent some features to not be supported or work as excepted
• improved accuracy of timestamp synchronization between networked machines using a new and more sophisticated algorithm
• message log now lists selected output file formats when starting export to help with debug reports
• clip offsets are now reset at the start of offline alignment
• new version available window now correctly adjusts number of decimals if needed
• console/headless version now logs the first 10 timestamp offsets to help with debug reports
• fixed issue on export range restriction when internal values were not available
• fixed issue when trying to change the clip color
• fixed a rare issue in the network triggering class by circumventing a possible crash in the underlying string library
• fixed cosmetic issue where selected clip was no longer drawing an outline
• fixed cosmetic issue where not all GUI elements were disabled during export
• fixed cosmetic issue where sensor ID in “Selected Sensor Settings” tab was 0-based, and not 1-based like in “Sensor Table”
• fixed issue where console version may not have read some of the more exotic sensor settings at startup
• fixed potential crash in body segmentation class
• fixed a crash when an Intel T265 was connected as these sensors don’t provide depth/pointcloud data
  • a mention in the message log is now added and these sensors will be skipped during initialization
• small speed optimization in normal and vertex color computation
• lots of small bugfixes, prevention measures, optimizations and potential memory leaks based on static code analysis
• driver download progress window is now ‘always on top’
• driver installer now writes to file during download, instead of flushing all data at the end
• updated to latest Azure Kinect v1.1.0 Body Tracker
• updated to latest Stereolabs ZED SDK v3.4.2
• updated Orbbec driver to latest SensorDriver V4.3.0.17
• updated to latest Intel RealSense v2.43.0
• extended beta to 1st of May (working on a big alignment module rewrite/update but not ready yet)

BETA v0.73

• support for MGear’s latest “Point Cloud Viewer and Tools” for Unity and it’s new v3 file format bringing:
  • streaming files directly from disk, good for large files
  • .bin parsing is now ~2-3x faster (on top of the previous 2x improvement)
  • now uses less memory (no more padding or extra array copy/clear)
• marker alignment now automatically uses the 30% best frames instead of always using all frames to help robustness and quality
• marker quality now takes into account how many consecutive frames a marker has been seen
• added a tip to the marker generator window to make sure to stick your printed marker to a rigid surface so it cannot bend
• added marker alignment statistics to the message log after it finishes to help debugging
• fixed issue when marker alignment was stopped an internal thread to check data kept running
• fixed issue in marker rotation time-interpolation code
• fixed potential memory leak with std::deque in various places
• speed optimization for texture upload to GPU for 2D views
• various small memory management speed optimizations in math library, marker tracker & network sensor initialization
• lots of small bugfixes, prevention measures, optimizations and potential memory leaks based on static code analysis
• added warning in GUI that small marker sizes are not recommended (unless the volume is small)
• Some internal changes to hopefully avoid issues with false positives on some anti-virus/malware heuristics
• updated to latest Intel RealSense v2.42.0
• extended beta to 1st of April

BETA v0.72

• audio class now reports warnings/errors to message log (instead of console window only)
• setting audio device now logs what format settings were actually used (since if an invalid format is set in the GUI it will try to select the closest valid settings)
• removed internal behavior of limiting number of network packets to process in one go as it could potentially prevent some crucial communication in rare occasions
• some minor cosmetic cleanup in headless/console mode output
• fixed cosmetic issue where markers in color view of sensor with mirrored color (like Azure Kinect) were drawn without mirroring
• fixed issue with headless/console mode not working and always starting full GUI on trial version
• fixed a potential (but rare) race condition for the GUI when passing bodies/markers to other classes
• fixed issue where network sensor didn’t start sending when GUI also had a local sensor
• fixed issue when running with multiple sensors on a network client
• fixed possible multi threading race condition when running with multiple sensors from a network client
• added additional debug info to the crash uploader to help isolate where exactly it crashed
• different way of compression used in installer, slightly bigger file but potentially better compatibility with anti-virus/malware scanners
• updated to latest Stereolabs ZED SDK v3.4.0

BETA v0.71

• added option to scale fused texture to smaller size
• fixed cosmetic issue where headless version would only display data of the first sensor
• fixed potential race condition crash when receiving markers from network sensor
• fixed issue with network machine with multiple sensors only sending data from the first sensor due to server not receiving all init packets due to max_packets restriction
• viewport text rendering will now be skipped on graphics cards / drivers that don’t support it to avoid a crash
• console/headless version now writes to a separate log file
• cleaned up message log output of connected sensors to be more compact
• updated to latest Intel RealSense v2.41.0
• extended beta to 1st of March

BETA v0.70

• added support to record audio
  • using any device supported in windows, not just depth sensors with embedded microphones
  • records to a WAV file
• added support to use WAV audio files on the timeline
  • can be files recorded along with BPC files or any other source
  • audio clips will be mixed during export and trimmed to the timeline export start/end settings
  • mixed audio can be exported to a WAV file
• added chapters to documentation on loading data into Unity, UE4 and 3D animation (or other) apps
• added options for different naming schemes for image sequences
• rewrote audio device handling and WAV writer
• rewrote internal handling of mute/solo states
• consolidated timeline / output setting reader so there is no duplicate code between GUI and batch processor
• moved “warn for overwriting” toggle to export panel to make it more easily accessible (instead of top menu)
  • this is saved/loaded with the timeline/output settings
• fixed a potential crash in the audio waveform display widget
• fixed issue where track mute/solo states weren’t being respected when number of tracks changed
• fixed issue where time sampling in Alembic files could be wrong resulting in files playing too fast or containing to few frames
• export range is now automatically restricted to range containing BPC frames only
• added additional checks to catch rare crash in marker detector
• start button in batch processor is now greyed out when there are no items in the queue
• updated several internal libraries to latest versions

BETA v0.69

• completely rewrote marker aligner
  • now has option to work with a static marker or moving marker
  • now comes with a brand new mathmatical solver that uses more data to optimize to the most optimal solution
  • ability to run a two pass solve (based on moving marker) for tighter alignment
  • ability to detect markers and do alignment based on clips on the timeline
    • this can be more flexible and more accurate than doing everything on live sensors
• implemented a new marker pattern/detector
  • old markers are still fully supported
  • new markers are labeled experimental
    • detection is usually a bit faster and more efficient, especially with moving markers
    • corner refinement should be a bit more robust
    • should be a bit more robust during movement
    • still labeled experimental (feedback is welcome)
• improved internals of marker detector to more intelligently use sensor calibration and detect marker type/size/mirroring
• rewrote how paper size and marker length are passed around
• added export preset for Holo CatchLight by Prometheus Vision https://assetstore.unity.com/packages/add-ons/holo-catchlight-plugin-pro-177405
  • allows you to use Holo CatchLight’s Unity playback integration
  • exports OBJ and JPG/PNG texture sequence
  • automatically uses compatible filename structure and settings
  • use Holo CatchLight’s ObjSeqConverter to convert the output to an MP4 file that can be used with the Unity integration
• added a LAS pointcloud exporter option
• added global offset
  • applies a position/rotation offset to all clips on the timeline
• added tooltip for SenseXR export
• added a Z-tweak multiplier option to BPC clips
• added a warning to the message log if multiple sensors are detected with one sensor not receiving any frames
  • suggesting that the computer may not have enough USB bandwidth for all the sensors
• added functionality to drag & drop thesefiles onto 3D viewport
  • alignment file, loads them onto live sensors or timeline (depending on what’s currently visible)
  • timeline file
  • output settings file
• added some missing tooltips
• Batch Processor default now has “Create Timeline Subfolder” enabled
• moved timestamp further to the left in the sensor table
• when Kinect v2 could not be initialized with synchronization it is now retried without synchronization
• fixed potential crash when loading calibration data from an old BPC file
• fixed a regression issue with mute/solo of track would hide the wrong clip in the 3D viewport
• fixed a regression issue where PNG output would save as JPG in last version
• fixed issue where timeline would not always resize correctly when loading new BPC file(s)
• fixed erroneous “Timestamps of sensors seems to vary quite a lot” message when GUI was displaying video or depth view
• fixed cosmetic issue where progress bars in batch processor were scaled 1 frame too large
• fixed cosmetic issue where “Generate Marker PDF” window wasn’t showing preview of the marker in release version
• fixed issue with Batch Processor not exporting SenseXR files
• updated to latest Stereolabs ZED SDK v3.3.3
• extended beta to 1st of February

BETA v0.68

• added export preset for Sense of Space’s SenseXR https://www.senseofspace.io
  • allows you to use SenseXR’s Unity playback integration
  • exports Alembic mesh and JPG/PNG texture sequence
  • automatically uses compatible filename structure and settings
  • from SenseXR’s top menu simply use File > Import > Alembic (.abc) to import the asses, it should automatically find the accompanying texture sequence
• fixed issue where triangles meshes had inverted normals
• changed how Alembic UVs are constructed to make it

BETA v0.67

• added an (advanced) option to blocklist sensors
  • allows user to specify a list of names
  • when initializing sensor names are checked against this list
  • if sensor name contains an item from the list it will be skipped and not connected
• these settings are now added to output settings files
  • global cropping settings (box & cylinder)
  • body segmentation settings
  • randomize points settings
• added additional check in image exporter to prevent crash when image data is somehow invalid
• fixed some memory leaks in body segmentation class
• fixed a possible crash in timeline GUI
• fixed issue with a few save dialog boxes stating “open” and not checking for file overwrite
• fixed issue with display of clips with large offsets on different tracks not displaying correctly
• fixed issue with batch processor not respecting time offset for clips
• fixed issue with batch processor not loading bounding box cropping Z values correctly
• internal sensor code & compiler settings now align better for trial/retail version
• changed to a more robust and standard way of handling keyboard shortcuts
• small improvements in memory managment
• updated to latest Stereolabs ZED SDK v3.3.1

BETA v0.66

• fixed issue with installer not copying some files needed for Orbbec sensors
• increased number of internal buffers
• small string conversion optimization in UDP network message creation
• added a Batch Processor
  • separate application that can be started from within the main app or Windows Start menu
  • reads one or more timeline files and exports to the various output file formats
  • options to overrule various things like output path/filename
  • allows you to process & export multiple files at once without the need of the base app

BETA v0.65

• fixed issue with initialization of Orbbec Astra sensors
• added functionality to save/load output settings to a file, instead of only saving/loading it with all timeline settings
• added mesh decimation setting to mesh export settings to reduce mesh density
• added option to switch meshing to quads or triangles (used to be quads only)
  • quads is faster to generate
  • triangles can produce slightly cleaner edges since they can be split in two diagonal ways
• small speed increase for meshing in quads mode
• added functionality to change clip color (used by visualization on the timeline and 3D viewport)
• added a warning at the end of marker alignment if your marker is relatively small in relation to sensor resolution and distance to it
  • and a suggestion to print a larger one for better results
• fixed a crash that could occur when exporting with textures enabled and then exporting again without textures
• fixed issue with texture path in OBJ material files being incorrect
• fixed issue with initialization of Orbbec Astra sensors
• fixed issue where exposure settings in GUI could get out of sync
• added a few small memory initialization fixes
• explicitly set heap & stack reserve site in compiler to 4MB now
• updated to latest OpenCV 4.5.0
• updated to latest Intel RealSense v2.39.0
• extended beta to 1st of December

BETA v0.64

• fixed issue in color to depth timestamp correlation used for recording BPC files
  • only seemed to occur when recording with video or depth window showing, not with 3D view
  • showed itself on Azure Kinect sensors but may have been prominent (to a lesser effect on other sensors too)
  • could result in frame stuttering between color & points when playing back in PointCloud v3
• fixed issue with timeline bounds becoming incorrect when removing all clips while playing and then adding new clips to the timeline

BETA v0.63

• added “Settings > Save Marker For Printing” option from the top menu so you can use it without connecting to sensor(s)
• added extra check in raw body data buffer copy
• fixed issue with paper size not switching to “custom size” after setting a custom marker length
• fixed a potential out of bounds error in marker aligner
• small change in internal buffer locking cadence
• “running multiple sensors on laptop” warning now only triggers if there are multiple local sensors (instead of local + network sensors)
• marker data for network sensors is now processed in background thread with double buffering, just like pointcloud data

BETA v0.62

• network classes now uses a new internal buffering method to avoid race conditions especially on slower machines
  • similar to what was already implemented for sensors and BPC reader in previous versions
• added feature to use gamepads to remote controll certain features
  • works with any gamepad that is supported by Windows (for example XBox controllers)
  • works with wired and wireless controllers
  • buttons can be configured to start/stop recording and alignment functions
• GUI now shows amount of free disk space
• console version now prints clock offset once connected to a server
• console now shows amount of free disk space (once recording has started and recording folder is known)
• more verbosity when variation of (network) sensor timestamps suspiciously large
• 2D texture views now highlight selected clip
• improved internal available buffer selection mechanism in BPC Reader
• fixed issue with BPC writer where it wouldn’t dump color timestamps in the lookup table when using Azure Kinect JPG stream
• fixed issue where marker length wouldn’t change as it was internally tied to the wrong GUI widget element
• fixed cosmetic issue with fps counters in sensor table weren’t properly colored for network sensors
• removed superfluous warning in BPC color loader
• default setting is to toggle network sensors ON in the GUI now
• message log now automatically switches back to originally selected tab after 5 seconds, if it was brought to the front on an error/warning

BETA v0.61

• implemented a more robust and deeper integrated way to synchronize clocks between network machines
  • if clocks are not synchronized this may break alignment and data fusion
  • also automatically detects when device vs machine timestamps need to be re-measured
• fixed a possible crash that occur at the end of alignment calibration on some setups
• fixed issue where disconnecting sensors while calibration was running didn’t properly reset the GUI button states
• fixed small cosmetic issue with rotaton and timestamp overlapping at times in console output
• added A1 and A0 paper size presets for markers
• disabled LowDelay and enabled KeepAlive on network sockets to hopefully help with network congestion on some systems
• sensor details are now printed to the log after a few seconds to give network sensors a bit of time to send their stream details to the GUI
• internally now forces the use of desktop OpenGL drivers (which was the default on almost all machines anyway)
  • overruling a QT_OPENGL environment setting that a user or other program may have set
  • a QT_OpenGL setting of “angle” could prevent some GUI widgets from being drawn

BETA v0.60

• fixed issue with Intel RealSense calibration printing out errors
• fixed a crash that could happen when initializing an Orbbec sensor
• cleaned up Settings menu in the top menu by moving items to Network and Sensors submenus
• fixed cosmetic issue with network bandwidth indicator displaying incorrect numbers
• improved Kinect v1 default calibration (unfortunately SDK for fetching data from device does not exist)
• improved Orbbec Astra default calibration (unfortunately SDK for fetching data from device is still broken and support staff is unresponsive)

BETA v0.59

• fixed regression issue with timestamps on network sensors not being adjusted properly anymore when synchronized with GUI
  • this could cause issues with alignment, pointcloud recording and data fusion with network sensors
• fixed some issues with Sensor Transform export
  • scale was 100 times too small
  • rotation offset was incorrect for default FBX camera pointing along local X axis
  • camera name now matches the name of the clip it belonged to
  • export no longer hangs after first frame when just exporting sensor transforms (and nothing else)
• fixed an issue with Azure Kinect internal depth frame not being cleared properly before filling
  • causing weird trailing on the bottom half of the image when loading a BPC file into PointCloud v3
• fixed issue with Kinect v1 timestamp correlation getting out of whack in some multi-sensor cases
• fixed issue with sensors from BPC file got a double transform in 3D viewport and FBX export (pointcloud was fine)
• added additional error checking for requesting calibration data for RealSense sensors as it could crash in some cases
• message that’s displayed when beta version has expired should be a bit more clear now and suggest downloading latest version
• rolled back to previous version or particle export library as latest version broke PRT export
• added option to top Settings menu to print calibration data for connected sensors
• implemented a completely new way to handle sensor calibration data
  • no longer needs lens calibration files
  • automatically switches to fetching details from drivers/SDK where possible
  • new internal development tools to validate and create default calibration data
• fixed issue with Orbbec driver detection for latest driver versions
• updated to latest Intel RealSense v2.38.1 SDK
• ongoing research & development work on (marker) alignment code and math, not ready for release yet though (upcoming versions may need you to print a new marker btw)

BETA v0.58

• quick fix release since regression in previous version could fail to detect license in some cases

BETA v0.57

• improved internal buffering of BPC reader
  • more robust against multi threaded race conditions
  • should no longer crash if resolution (or some other things) change between frames (for example when switching Azure Kinect clip between lowrez / hirez)
• fixed issue with color-depth transform not being correct for Azure Kinect BPC clips (for lowrez/hirez switch)
• fixed some missing data in network packets for markers
• fixed issue with marker set origin
• fixed issue where single sensor marker length and paper size weren’t properly connected to internal functions
• moved single sensor marker length to advanced settings (just like multi sensor already was)
• added a mention to the license info window that it will only display v3 license info
• updated to latest ZED SDK v3.2.2
• updated particle export library to latest version
• lots of research & development work on (marker) alignment code and math, not ready for release yet though (upcoming versions may need you to print a new marker btw)

BETA v0.56

• added a scale multiplier on the export tab to adjust the scale for mesh & particle exports
• added a toggle to enable/disable UV export for Alembic files (used to always be enabled in previous versions)
• added option to write vertex colors to Alembic files (this may not be supported by all 3D apps)
• some small memory & speed improvements in Alembic exported
• fixed issue with Licensing > Show License Info not properly working on beta before

BETA v0.55

• made BPC-reader and network classes more robust against trying to decompress buffers with invalid size
• multi-core optimizations in the sensor frame processing (depending on CPU and core count up to several milliseconds per frame)
• multi-core optimizations in the network frame codec (depending on CPU and core count up to several milliseconds per frame)
• fixed issue with the new buffering causing pointcloud/floor alignment/refinement not to be able to fetch the latest buffer at times
• fixed some issues with pointcloud alignment (may still not be optimal, working on improving this for the future)
• fixed issue with floor leveling during pointcloud alignment using an old and much less robust algorithm than “Refine Floor Plane” did
• fixed issue where floor refinement didn’t work properly for multiple sensors anymore
• pointcloud alignment is now more verbose in the message log
• removed “Slow” preset for pointcloud alignment/refinement as it didn’t perform any better than the standard “Fast” one
• last message log file is now moved to a _old folder at startup so we always keep the current and previous log for debugging / crash reporting purposes
• updated to new compiler profiting from new optimizations
• updated a bunch of internal libraries to the latest versions
• updated to latest ZED SDK v3.2.1

BETA v0.54

• added additional check in GPU load measurement to prevent crashes on AMD GPUs where measurement doesn’t work
• optimized internal buffering algorithm
• fixed issue with recording lossless color for Azure Kinect causing a crash at the start of the recording
• fixed issue where internally BPC file didn’t store ZED sensor model correctly
• fixed issue where app could get stuck in a loop re-writing registry settings in certain GUI configurations resulting in slowdown
• last message log file is now moved to a _old folder at startup so we always keep the current and previous log for debugging / crash reporting purposes

BETA v0.53

• new memory buffering algorithm for live sensors & BPC file loader to prevent race conditions
• fixed some potential issues with Intel RealSense sensor model detection and potential initialization crash
• fixed issue where ZED settings would also show up for other sensors on selected sensor tab
• switched to slightly less aggressive compiler optimization regarding instruction set as it could cause a crash with pointcloud alignment
• updated to latest Intel RealSense v2.36.0 SDK

BETA v0.52

• quick fix release since I forgot to enable the StereoLabs ZED GUI buttons, doh

BETA v0.51

• optimized 3D pointcloud rendering speed
• added a new and more accurate, but also slower, body segmentation algorithm
• progress bar now also lists current frame number during export
• message is printed to the log when a new firmware is available for Azure Kinect sensor
• several small bug fixes
• fixed potential bug in network server
• fixed issue with marker detection being unreliable on previous version on Kinect v2
• fixed issue with network sensor FPS readings not updating correctly in sensor table since previous version
• fixed multi-threaded race condition in console/headless version when sending network data
• updated to latest Azure Kinect drivers/SDK v1.4.1
• updated to latest Intel OpenVINO SDK

BETA v0.50

• added support for StereoLabs ZED sensors (v1, v2 and Mini should work)
• fixed an issue with 3D viewport not drawing anything on some AMD graphics cards
• fixed an issue where loading Azure Kinect clips got broken in the last version
• slightly tweaked internal marker detection parameters
• switched to using device lens intrinsics from drivers/SDKs that support it

BETA v0.49

• added support for latest Intel RealSense L515 LiDAR sensor
• added option to top menu to enable/disable playing of beeps at recording/alignment start/stop
• added Remote Triggering option
  • all Brekel apps have this option to synchronize recording start/stop/filenames
• changed how internal buffers are handled in BPC loader to prevent race conditions
• when no sensor is selected Selected Sensor Settings - ID now reads “Nothing selected in Sensors Table”
• added option to Sensor Settings tab to allow for setting if current sensor is primary or not
• fixed crash in RealSense color to 3D lookup (could manifest during marker detection for example)
• extended beta license to 1st of August

BETA v0.48

• changed how FPS is calculated for display in the GUI
  • used to be based on system clock timestamps in the processing loops
  • now is based on the frame timestamps, which (on the SDKs that support it) better reflect the real device fps
• added a GPU usage percentage readout (NVidia only) to the bottom right of the GUI
• the “Install Drivers” options now read “Download & Install Drivers”
• added a warning message if after 15 seconds no data has been received when downloading drivers
• the sensor table and display settings are now greyed out at startup to pursuade new users to look at the “Connect Sensors” button
• increased “Connect Sensors” button a bit to help new users find it
• when the user hasn’t connected sensors in 10+ seconds after startup a friendly reminder hint is displayed in the message log
• small cosmetic tweak for empty lines in the message log
• updated to latest Orbbec SDK v2.1.1

BETA v0.47

• lightened the icon colors a bit
• Hyper-V and Virtual network adapters are now considered when selecting a suitable network adapter
• added some additional compiler optimizations
• prepared licensing system for a wider release
• prepared Trial version

BETA v0.46

• no longer crashes when connecting a sensor on a CPU without AVX2 instruction support
• fixed issue with Azure Kinect pointcloud being cropped in previous version
• Azure Kinect resolution change now takes effect on the next sensor connection to avoid possible crash
• you can now stop a running pointcloud alignment/refinement by clicking the button again
• made the sensor stall detector a little more conservative
• updated to latest Intel RealSense SDK v2.34.0

BETA v0.45

• small improvement on color image debayering on Kinect v1 for both speed and edge quality
• some fixes to help prevent deadlocks
• additional check in BPC loader for invalid packets
• missing sensor drivers are no longer logged as warnings/errors
  • this would bring the message log to the front for users that may not be interested in some sensor types
• at startup “connect sensors” button will now slowly blink to attract attention from first time users :)
• console version now also prints CPU & GPU info at startup
• CPU info now also includes AVX/SSE support
• extended beta license to 1st of July

BETA v0.44

• fixed possible crash for Azure Kinect when switching to Video tab
• added a Processing Time readout in the selected sensor tab to help find bottlenecks
• added a warning when running on a laptop not connected to power
• added a warning when running multiple sensors on a laptop
• added a Slow/Fast preset for pointcloud based alignment/refinement (previously was always fast)
• tab in bottom dock now automaticaly switches back to timeline when export finishes
• fixed an issue when muting/soloing a track scrubbing the timeline would show/hide the wrong track data

BETA v0.43

• added option to toggle message log sounds for errors & warnings
• fixed issue where video textures were horizontally flipped (and didn’t map correctly on mesh UV coordinates)
• sensors are now set to manual exposure by default
• improved timestamp accuracy for Kinect v1 / Kinect v2 / Azure Kinect and RealSense sensors
• added a stall detection background thread that checks for stalled sensors/solvers/etc and logs a warning when detected
• fixed an issue with marker detection on Intel RealSense sensors
• updated some internal libraries
• updated to latest Orbbec SDK
• extended beta license to 1st of June

BETA v0.42

• added hint where to find Azure Kinect sync ports when running multiple sensors without sync cables
• added first revision of documentation in PDF form
• added additional internal smoothing for floor detection
• updated to latest Azure Kinect SDK v1.4.0
• updated to Intel RealSense v2.33.1

BETA v0.41

• fixed issue with 3D viewport camera rotation
• text drawing in console is now more efficient
• text drawing in console doesn’t flicker anymore
• text drawing in console now uses warning colors for
  • high CPU / memory usage
  • low frame rates
• marker tracker now runs on it’s own thread instead of on the sensor thread
• added additional checks in data fuser to prevent invalid numbers
• added additional check in Alembic exporter to prevent invalid UVs
• added additional protection against multi threaded race conditions in drawing classes
• minor multi threading optimization in data fuser
• extended beta license to 1st of May

BETA v0.40

• fixed potential crash when connecting to a Kinect v2 sensor while it couldn’t access one of the requested streams
• fixed potential crash in several exporters by adding a color out of bounds check
• fixed potential crash in texture exporter
• fixed issue where cropping bounding box / cylinder would dissapear when changing some other drawing settings
• fixed issue with video exporter mirroring
• fixed issue where color mapping to pointcloud could be broken on Kinect v1 sensors
• fixed issue where console would state Brekel Body instead of PointCloud
• fixed issue where console’s close button was disabled when running in headless mode
• fixed issue where Kinect v2 sensor would freeze after initialization in headless mode
• fixed issue where video/depth windows would not draw data after the first sensor connect
• speed optimizations for several internal math functions
• speed optimizations for Kinect v1, Kinect v2 & Azure Kinect sensors for multi-core machines
• speed optimizations in BPC reader
• “Fused Data” drawing settings are now only displayed when live sensor data is displayed
• “Live Sensor - Color Exposure Settings” are now only displayed when live sensor data is displayed
• marker based aligner now:
  • uses best quality markers that got collected instead of rejecting potentially bad ones at the incoming stage
  • quality is determined by the angle to the sensor and velocity (lower is better)
  • user can tune the percentage of markers used
    • 0.00 is all markers used for alignment regardless of quality
    • 0.99 only the 1 percent best quality markers used for alignment
  • adjusted the defaults for “Detect Every Nth Frame” and “Align Every Nth Frame” to 5 (was 10)
• added green screen options in clip settings
  • allows to crop out pixels of a green screen
  • has option for spill correction to help color correct greenish pixels that reflected light from the green screen
  • has option to erode the mask a few pixels to clean up the edges

BETA v0.39

• small tweak for non-additive pointcloud drawing
• added option to the GUI for to shrink body segmentation mask
• fixed issue with crash report generation/upload
• fixed issue where Kinect for XBox360 sensors depths were getting clipped
• fixed issue with settings being saved too often
• fixed issue where edge filtering in BPC loader post processing got broken and stopped working
• implemented a new architecture for brand/type specific sensor settings
  • dynamically adds controls to GUI
  • sends/receives them for network sensors
• added exposure control for Kinect v2, Azure Kinect sensors & RealSense sensors (seems not supported on Kinect v1 & Orbbec)
• added tilt motor control for Kinect v1 sensors
• added near mode control for Kinect for Windows v1 sensors (not supported on Kinect for XBox 360)
• added color offset setting for Kinect v1 & Orbbec sensors
• added color mirror setting for Orbbec sensors
• added emitter and filtering settings for Intel RealSense sensors
• added “custom size” option for marker size
  • in case you printed them and the measured size is slightly different
• improved mouse-over tooltip for alignment start/stop buttons for additional clarity

BETA v0.38

• exposed body segmentation to the GUI
  • you can now enable this to isolate bodies and remove the background automatically
  • based on a sensor agnostic, deep learning detection and segmentation approach
• improved overall body segmentation quality and stability
• CPU/Memory usage indicators now turn orange/red on high usage
• fixed issue with floor detection not being applied to body segmentation internally, which could cause less accurate results for refinement
• fixed memory leak in body segmentation code
• fixed issue where transform offset rotation Y & Z were coupled in the GUI
• fixed a few issues where some of the newer settings didn’t save/load with the timeline settings correctly
• added option to crop using a cylinder shape (beside already existing bounding box)
• added option to limit marker detection to every n-th frame to reduce CPU overhead
• selected sensor’s pointcloud now always draws on top while others are dimmed
• sensor frustums now draw in the same color as the sensor/pointcloud
• markers now draw a line to the sensors that detected them in the 3D viewport
• bodies now draw a line to the sensors that detected them during alignment
• increased internal threshold for marker interpolation time
• alignment now reports to the GUI:
  • which sensors were aligned to which
  • how many markers were used for the alignment
  • also logged when the alignment is finished
• added a popup dialog when loading alignment file which allows to specify per sensor what data to load from the file
  • you can specify to not load data for a specific sensor
  • you can remap in case current sensor names differ from what is in the file
  • if matches are found a one-to-one mapping will be the default (you can hit enter to immediately load)

BETA v0.37

• 2D views now include sensor ID numbers along with their names
• reduced clutter in commandline window during initialization of Azure Kinect sensors
• fixed issue with sensor drivers not being recognized
• fixed issue with sensor driver install options not showing up under settings in top menu
• added cropping to post processing settings (applied to all sensors at once in global space)
  • removed per-sensor post-transform cropping from clip settings
• added randomize to post processing settings (applied to all sensors at once in global space)
  • removed per-sensor randomize from clip settings

BETA v0.36

• added support for PLY in binary format (as used by Keijiro’s PCX for example)

BETA v0.35

• updated to Intel RealSense SDK v2.32.1
• updated to Alembic v1.7.12
• updated to FreeImage v3.18.0-8
• message are now not only displayed in 3D viewport but also 2D viewports and bottom status bar
• improved internal body segmentation
  • helps for pointcloud alignment & refinement, especially for Azure Kinect
• when selecting a sensor the tab on the left now auto switches to “sensor settings”
• when selecting a clip the tab on the left now auto switches to “clip settings”
• fixed issue where tilt settings didn’t show up in the GUI for Kinect v1
• fixed issue where RealSense specific settings didn’t show up in the GUI
• sensor IDs in table now start with 1 instead of 0
• primary and selected sensors are now indicated in the 3D & 2D viewports and sensor table
• fixed a bug where sensor names would dissapear in 2D views after selecting one
• fixed an issue where infrared texture would draw as black in 3D viewport for Kinect v2
• implemented better way to get a unique id/name for Kinect v1 & v2 devices
• updated internal libraries
• started digitally signing executable to help alleviate anti-virus false positives

BETA v0.34

• small efficiency improvements in network encoding/decoding
• investigated GPU (NVidia & Intel) compression/decompression
  • leaving things to CPU for the moment as the speed is comparable and it’s compatible with all machines
• created a new algorithm for segmenting humans from pointcloud data without the need of joint data
• implemented a new pointcloud refinement method which is faster than the old one and now multi-threads across sensors
• implemented a new pointcloud alignment method
  • does an initial (rough) alignment based purely on pointcloud data (no bodies/markers)
  • will ask you to stand in the middle of the volume (an assymetric pose can help)
  • this is dependent on pointcloud overlap and may not be as robust as marker-based alignment for all setups
  • can automatically perform pointcloud refinement in succession for a tighter fit
  • can automatically perform floor leveling
  • not fully robust yet (see known issues) but a new version needed to be released due to expiration of the previous beta
• rewrote the floor detection and segmentation algorithm for refinement based on floor to be more robust
• rewrote internal timer so code can be reused more easily
• removed PCL dependency
• now sets OpenMP environment variable to prevent 100% CPU usage on some machines in some cases
• marker alignment now reports how many matches it found in the log
• extended beta license to 1st of April

BETA v0.33

• network communication read buffer is now set to 1MB instead of automatic
• network buffers are now polled and read at set intervals isntead of automatic
• network low latency flag is now enabled
• network keep alive flag is now disabled
• low frequency mode check is now disabled in network sensor
• fixed bug where stream fps wasn’t correctly used from network sensors
• fixed issue where changelog wouldn’t display when new version was available
• small speed improvements in sensor & network processing calculations

BETA v0.32

• fixed cosmetic issue displaying driver download link when downloading from backup URL
• fixed issue when loading calibration files sometimes data was applied to the wrong sensor
• fixed problem where internal ids of multiple Azure Kinect sensors could become incorrect when using sync cables
• updated to Azure Kinect SDK v1.3.0

BETA v0.31

• fixed an issue where Alembic files had a corrupt time sampling setting when the export start frame was 0
• fix for a rare multi-threaded race condition in the BPC loader
• cleaned up some threads on program exit
• added option to select preferred network adapter (Top Menu > Settings)
  • this will be used at startup of the GUI or headless version of the app
• updated to latest Orbbec SDK
• extended beta expiration date to 1st of February 2020

BETA v0.30

• changed some of the pointcloud drawing default settings to be better suited for textured pointclouds
• when downloading a driver links no longer display %20 for spaces
• 2D video widgets make a deep copy of incoming data again
• memory managment now drops frame data copy and sending it to GUI if that is still busy processing the previous frame
• updated Orbbec driver link to astra-win32-driver-4.3.0.10
• fixed cosmetic issue with sensor name/resolution text in 2D widgets when selecting a sensor
• fixed issue with network sensor saving it’s resolution to the registry under it’s network name instead of local sensor name
• fixed issue with the marker aligner not sending it’s results to the GUI anymore in release version
• fixed issue with network client timestamps not synchronizing correctly in release version which could cause:
  • issues with potential sensor connection loss
  • issues with display flicker
  • issues with pointcloud recordings

BETA v0.29

• changed how internally frames from the sensor processing threads are shared with the GUI
  • to prevent some rare crashes that occur when running the software for 20 minutes on some machines
• web links and changelog pointed to Body v3 pages, fixed now
• increased minimum network packet size for internal processor for better stability
• network sensors with disabled pointcloud streaming now continue to report fps & timestamp to GUI
• made top menu background a little darker for better contrast with rest of the app
• removed “alignment refine” tab (for now) as it relied on body tracking joints (working on some new stuff for that)
• removed the need for some internal memcopies for frames, this should increase speed and efficiency
• removed the need for some internal multithreading mutex waiting, this should increase speed and efficiency
• fixed issue with flickering in 3D & 2D widgets for network sensor data
• fixed issue where depth from network sensors didn’t display in GUI anymore after a previous update
• fixed issue where sensor models were drawn reversed
• fixed issue where cropping values for live sensor had Z values reversed
• fixed issue where deselecting timeline clips didn’t always redraw
• fixed issue where wrong pointcloud would be shown/hidden when clips were offset on timeline
• fixed issue with setting transform on a network sensor did not affect the right sensor if there were multiple sensors on a network client
• fixed cosmetic issue with network bandwidth display in the console
• fixed cosmetic issue with 2D viewers not always displaying sensor name
• fixed cosmetic issue with sensor frustums not drawing for network sensors
• changed sensor initialization sequence for headless/console version
  • should now resemble GUI version
  • should now respect resolution settings read from registry
• timeline controls:
  • CTRL + Left Mouse Click to select clip
  • Middle Mouse Click to deselect clip
  • SHIFT + Left Mouse Drag to slide clip and adjust in/out points

BETA v0.28

• fixed a bug where marker tracking for Azure Kinect in InfraRed mode stopped working after a previous update
• fixed a bug where some sensors had a marker offset in 3D space
• fixed a bug where transform offset rotation Y & Z were acidentally coupled in the GUI
• fixed a (rare) potential crash in the 3D widget on startup
• fixed issue where sensor frustum’s were inversed
• fixed issue when using network sensors only the first sensor would toggle marker tracking on/off
• fixed issue when using network sensors where tracked marker data wasn’t applied to the correct sensor in the GUI app
• fixed issue with network sensor markers being rejected too quickly due to sensor transform being sent as position/rotation instead of matrix
• network communication now always sends full transform instead of position/rotation
• when typing values into spinbox widgets internal value is only updated when editing has finished
• restricted Azure Kinect messages to critical only
• added a warning when using multiple Azure Kinect devices without using sync cables
  • includes link to Microsoft documentation web page on how to install such cables
  • https://support.microsoft.com/en-us/help/4494429/sync-multiple-azure-kinect-dk-devices
• added new commandline toggles for enabling/disabling mutex usage to control multi-threading -use_mutex_3d turns mutexes ON for 3D view -no_mutex_3d turns it off ( default = ON) -use_mutex_video turns mutexes ON for video/depth views -no_mutex_video turns it off ( default = ON) -use_mutex_sensor turns mutexes ON for sensors -no_mutex_sensor turns it off ( default = OFF) -use_mutex_network turns mutexes ON for networking -no_mutex_network turns it off ( default = OFF)
• added multi-threading deadlock detector that can hopefully help identify program hangs
  • a message will be printed to the black console window
  • a message will be added to the log file to your “\Documents\Brekel\Brekel_Body_v3.log” file
  • please drop an email to beta@brekel.com with a copy of the warnings as these are very hard to reproduce as it doesn’t always occur on every system/setup/workflow

BETA v0.27

• fixed a bug where mute/solo was hiding/showing the wrong clips in the 3D widget and exporter
• tooltip and statustip descriptions now mention “locally connected sensors” explicitly
• when a certain sensor type was not found after trying to connect the checkbox is now automatically disabled in the GUI
  • so sensor discovery will be more efficient the next time only focussing on existing sensor types
• added option under Settings menu to force saving settings
  • as usual settings are saved to the Windows registry and automatically loaded at startup and saved when the app exits
• internal changes on how last used resolution is loaded & applied at startup
• consolidated color/IR resolution into video resolution
• app will now ask to upload latest crash report file at startup if it finds one
• improvements to marker tracking accuracy
• updated internal libraries

BETA v0.26

• crash uploaded was mangling description text on some machines
  • switched to new way of adding text to the report
  • hopefully this helps as I can’t reproduce it on any of my machines
• added exporter for raw depths in PNG file format (uint16 millimeter units)

BETA v0.25

• fixed a crash in the crash uploader tool that could occur on some machines (isn’t that ironic)
• UVs were flipped for geometry output formats, fixed now
• fixed cosmetic bug where changing export start/end frames didn’t update drawing in the timeline
• fixed issue where timeline zoom wouldn’t update correctly at startup until current frame was changed
• video output now automatically scales if frame size changes during the timeline (scaling to the initial size)
• lowrez/hirez toggle for Azure Kinect clips was hidden in public version
• disabled network multi threading mutexes by default as it turned out to be that a previously fixed crash was not in the networking code after all
  • can be re-enabled using this commandline flag: -usenetworkmutex

BETA v0.24

• 2D views now draw clip name and resolution
• sensors in 3D view now show clip name
• 2D views now clean up when the number of tracks in the timeline change
• GUI now displays a message in the viewport while opening a BPC file
  • especially handy at startup when waiting for a few files to load from the previous session
• fixed a crash that could happen during export
• added option to skip creating a bug report when a crash is detected
• lowrez/hirez mode for Azure Kinect files now actually switches
• added option to directly upload a bug report along
  • your name/email so I can contact you (optional)
  • a small description on when the bug happened (mandatory)
  • a description explaining how to reproduce the bug to help me recreate and debug it (optional)
• added Mute/Solo buttons for timeline tracks
  • mute hides all the clips on that track
  • solo only shows the clips on that track
  • mute/solo applies to both the 3D view as well as export
• updated internal libraries to latest versions

BETA v0.23

• added range slider widgets to the cropping settings
• worked on the range slider widget internals
  • now prevents invalid values
  • can now be paired with spinboxes
  • better integrated painting
  • dragging is more intelligent depending on where the user clicked
• Z cropping was inverted in the GUI, fixed now
• added some additional protection against multi-threaded race condition crashes
• app now tries to keep track of which driver/SDK version you installed
  • tracks driver/SDK installed from any Brekel v3 app
  • compares to latest supported driver/SDK version
• updated to Azure Kinect Body Tracker Preview v0.93 for improved tracking stability
• added compression and binary/ascii options to PCD exporter
• Azure Kinect sensor warning/error messages no longer play sound
  • especially during startup when acquiring sync it could produce a cascade of warnings.
• changed the mechanism on how the app checks if sensors have stopped recording
  • should prevent a signal not getting across and the GUI not resetting back ready for a new recording, especially on fast machines
• improved quality of “Remove Points Outside Color Lens” filter

BETA v0.22

• Orbbec sensors should now be found and should work, after recreated the build project
• removed “Single-Sensor License” as it is not relevant, beta’s have no restrictions
• added additional protection against invalid packets in BPC reader
• added additional protection against invalid network video packets
• fixed an issue with some sensor network packages not being initialized properly which could cause a crash on first connection
• fixed an issue that could make the GUI crash on recording end in some cases
• fixed an issue when connecting to a headless client that got disconnected before
• small memory & performance improvements for network video compression/decompression by caching some things
• console/headless version now prints the last few messages from the log to the console
• errors/warnings from Azure Kinect drivers/SDK are now passed through to the message log
• cleaned up some leftover debug messages

BETA v0.21

• fixed issue Azure Kinect color in BPC files due to internal memory manager not passing compressed buffer
• fixed an issue where some old BPC files from v2 would not load any points
• fixed small cosmetic issue where Kinect v2 sensor would report an infrared framerate (when it’s actually using the color stream)
• fix to prevent timeline bounds to cross
• hidden a few GUI controls of unfinished features
• cleaned up the message log on sensor initialization a bit

BETA v0.20

• Initial public beta release