City CAD Models for Holicity Machine Learning

3D City CAD models represent a cornerstone dataset for HoliCity, a city-scale 3D dataset with rich structural information. Currently, this dataset has 6,300 real-world high resolution panoramas that are accurately aligned with the CAD model of downtown London with an area of more than 20 km2. Working on a joint project by University of California (Berkeley), Stanford University and Bytedance Research, Palo Alto, California, USA, researchers Yichao Zhou, Jingwei Huang, Xili Dai, Linjie Luo, Zhili Chen and Yi Ma developed a platform which aims to be an all-in one data platform for research of learning abstracted high level holistic 3D structures that can be derived from city CAD models.

3D structures that can be derived from city CAD models, e.g., corners, lines, wireframes, planes, and cuboids can be supporting real-world applications including city-scale reconstruction, localization, mapping, and augmented reality. Thanks to the unprecedented city-wide accurate alignment of the 3D CAD models of London and its respective pangrammatic photos also benefits low level 3D vision tasks such as surface normal estimation, as the surface normal extracted from previous LiDAR-based datasets is often noisy. Accurate and detailed 3D city models in CAD supplied by AccuCities played an important part in the ability to develop this dataset.

“The problem with data-driven approach to machine learning is that you absolutely do need the highest quality data – and lots of it.” said  Yichao Zhou, researcher from UC Berkeley. “Methodology, experimenting and coding are of course integral parts of this process, but in my opinion a large area of CAD city model and corresponding RGB imagery are an absolute must. That’s why it was so welcome that AccuCities supported this project with their 3D London city data. The accuracy and detail of the 3D city model was absolutely crucial for the method to work. There is still much to be done, but the project clearly illustrates that it is possible to teach the system so it can make better plane detection and depth estimation. We can also conclude an encouraging level of transferability of the systems pre-trained using Holicity.”

AccuCities offers generous 3D city CAD samples for London, Bristol and Cardiff in the UK and Dublin in Ireland. These can be downloaded directly from AccuCities website. Researchers interested to use, develop and enrich the HoliCity dataset can request up to 1 square kilometer of 3D city CAD data from our library or purchase the entire dataset used by HoliCity project. Get in touch to discuss.

Researchers now conduct experiments to demonstrate the applications of HoliCity, such as predicting surface segmentation, normal maps, depth maps, vanishing points, as well as test the generalizability of methods trained on HoliCity and other related datasets.

Disclaimer: The street-view images are owned and copyrighted by Google Inc. The refinement of images’ geographic information and rendering of CAD models are done by UC Berkeley. For any commercial or profitable usage of the datasets, one must seek explicit permissions from each of these rightful owners.

Acknowledgement: Holicity is partially supported by research grants from Sony Research, Berkeley EECS, Berkeley FHL Vive Center for Enhanced Reality, and collaboration with Bytedance Research Lab (Silicon Valley).

Read the published paper below or visit https://arxiv.org/abs/2008.03286 to read/download the paper.