Posts by: pierotofy

The road to run everywhere: ARM64 (Apple M1)

 

OpenDroneMap is now compatible with the Apple M1 (no, not with this arm). The community came together to fund this feat. Thanks to everyone that contributed!

What was challenging about this?

Apple M1 speaks a different language than Intel chips (the mainstream chips that most people’s computers have). This language is a dialect of ARM. Apple has added a piece of software to guarantee some compatibility with applications not specifically built for its new chip, but the compatibility layer fails with software such as ODM that has many optimizations and low-level functions.

The steps to a port are always more or less:

  1. Try to build the software on the new architecture
  2. Get showered with errors
  3. Fix a few errors
  4. Repeat from 1 until success
  5. Re-test on every other architecture (in case you’ve broken something in the process)
  6. Automate the build process

Some really strange bugs came up during this process (like this one or this one). We upgraded the base layer of the software to be based on the cutting edge Ubuntu 21.04, since certain packages we depend on aren’t available for ARM. This broke our deployment to Snap. So we had to fix that with some conditional build logic. In the process our Windows build broke. We finally got everything ready for building and deploying, but the build times were now exceeding 6+ hours (up from 2+), which is a limit on GitHub. So we setup our own GitHub builder node, which is faster and enjoys higher time limits.

A challenge was getting PostgreSQL + PostGIS running. To guarantee compatibility with previous releases and a smooth upgrade experience for long-time users, WebODM’s docker images need PostgreSQL 9.5. As we plan for a future database migration / update which will require us to include multiple Postgres installations in the same database image, we opted to build from source both packages (PostgreSQL 9.5 is no longer included in newer Ubuntu releases).

After several days (and lots of coffee), everything came together!

So there you have it! From the command line:

git clone https://github.com/OpenDroneMap/WebODM
cd WebODM/
./webodm.sh start

Will get you up and running on the Apple M1 (and on other ARMv7 devices such as newer Raspberry PIs too)!

What’s next? Maybe a sans-docker port to MacOS? We’ll see. Stay tuned.

On Code and People

 

I’m proud to to announce that as of last week, our esteemed community member, long-time contributor and all-around champion of open source Saijin_Naib has become the first non-developer paid contributor to be sponsored by UAV4GEO.

He’s no stranger to OpenDroneMap. He has been involved with the project for a long time and has demonstrated incredible enthusiasm across these years while helping people, rallying for fundraising rounds and advocating for a world with more free and open source software. UAV4GEO is delighted to be able to support his efforts to continue doing what he has been doing all along: helping people with OpenDroneMap. He will also spend time to expand and improve the documentation across all projects and provide hands-on learning experiences to the community. We look forward to see what he will bring next.

I think every open source project one has to choose between two priorities: code or people. If you want to invest into building features, you invest in software developers. If you want to build a community, you invest in the community. With finite resources, you cannot prioritize both. I think a strong community, with it’s diverse set of experiences and ideas, over the long-run, will lead to better code. And for communities to thrive, they need people like Saijin_Naib that will jump in to lend a hand when things get confusing or when somebody needs some guidance.

So, congratulations Saijin_Naib on your new position of IT Specialist and Community Manager! See you on the forum.

The road to run everywhere: Native WebODM on Windows

 

One of the cool things about open source software is that it has a tendency to eventually run everywhere. If you have the source code available and certain constraints allow it, with sufficient time a port can be made to different platforms. This is not always the case with proprietary software, where the vendor decides which platforms to support.

In the beginning, ODM could only run on Ubuntu Linux. Later, we opted to leverage docker, which is an amazing (yet confusing) technology we use to bring many of the OpenDroneMap tools to other platforms such as MacOS and Windows. The docker abstraction allows us to focus on building the software without worrying about the platforms. If you can install docker, you can run OpenDroneMap software. Simple. But not without issues. Especially on Windows, docker has several quirks.

We initially delayed efforts to make a native port to Windows for two main reasons:

  • To focus developer time on quality of outputs, speed and features.
  • Previous attempts at making a Windows native port (2 of them across 3 years) failed to yield a working prototype (and lots of lessons learned).

Third time is the charm. I’m pleased to announce that starting today, you can run ODM, NodeODM and WebODM entirely natively on Windows. Even without docker. So where/how can you install it?

This was no easy feat. Let me tell you a bit about the challenges we encountered:

  • ODM has a several primary dependencies (OpenCV, OpenSfM, mvs-texturing, PoissonRecon, PDAL, untwine, entwine, GRASS, OpenMVS, dem2mesh, etc.). These dependencies in turn have in some cases lots of other dependencies.
  • Most of these dependencies were/are written with non-Windows systems in mind.
  • Path handling requires special care on Windows. For example, you cannot create symbolic links on Windows, unless you have administrative permissions (that’s a no go). Paths have backward slashes instead of forward slashes. Single quotes are no good for wrapping paths with spaces in Windows shell commands and functions such as pipes.quote in Python will use single quotes. CTRL+C will prompt the user for a confirmation to terminate a process (instead of just terminating the process), which will hang forever in headless processes like NodeODM.
  • Building Python packages from source code (which ODM requires) leaves several hard-coded paths in our packages, making deployment to other machines tricky.
  • Python virtual environments weren’t meant to be used for deployment (they also have hard-coded paths). But ODM needs a dedicated Python environment to run.

On the plus side, we had already proven last year that the user interface could work natively on Windows:

So all that remained was to get the ODM and NodeODM components to work too.

  • We set off to build each major dependency (and contributed several patches to other upstream FOSS projects in the process). We used a tool from Microsoft, VCPKG, to help us build several secondary dependencies for the projects that needed them. This was probably the hardest and most time consuming task. Libraries missing headers, architecture set instructions problems, pre-processor define issues, runtime dependencies not being found, path resolution issues, CMake variable propagation troubles. The list goes on. But divide and conquer, lots of coffee and one by one, we managed to build them all (the entire saga can be read here, thanks to all the community members that followed the process and gave encouragement along the way).
  • We refactored/cleaned up the directory structure and eliminated the build directory in the process, allowing us to keep all of our dependencies in a single SuperBuild/install directory. This made it easier to package the software in subsequent steps.
  • We fought the idiosyncrasies of Windows Batch scripting and the Windows shell with some clever tricks to enable a user to simply run: run d:\my\dataset from a command prompt (the ODM Console) and allow to cancel it via a CTRL+C.
  • We hot-patch several hard-coded paths as part of the ODM startup process to make sure our Python modules work as expected.
  • We rewrote the cutline computation algorithm in Python instead of using GRASS, which allowed us to shed some space requirements (and improve performance).
  • We wrote an installer script using InnoSetup (another awesome FOSS) and we got pretty clever to automatically sign the final installer with a code signing certificate (otherwise Windows and other AV software would complain).
  • Lots of testing, which yielded its own set of problems, like undefined behavior in MVE/libjpeg causing a file descriptor leak due to C/C++ exception throwing, which were resolved after some debugging.
  • With ODM working, we turned the page to NodeODM. Luckily, NodeODM is a relatively simple piece of software and is written in portable NodeJS. Save a few minor and not so minor issues, we managed to use nexe to generate a portable executable for Windows. Here we found out that Windows exit codes do not propagate well unless you explicitly call exit on every Batch script (but that breaks console-based workflows), so we wrote some clever workarounds.
  • With NodeODM and ODM both working, the path to full Windows native WebODM was finally possible. Using the same technique that we explored last year, we generated a single executable setup that installs everything that you need. We finally improved the Desktop Mode capability of WebODM to give a more native feel to the application, which remains a web-based application running under an Electron wrapper.
  • We finally built an automatic updater system to allow people to check for the latest version of the software.

So there you have it: Windows native WebODM.

What does this mean for docker? Nothing. Docker is and will continue to remain a core part of our toolset and deployment mechanism. This is simply another way to run WebODM that we hope you will enjoy.

Automatize GCP image coordinate collection

 

This is a guest post authored by Zoltán Siki, the creator of the awesome Find-GCP project.

Have you tried to create the gcp_list.txt file for a project with hundreds of images? It is a time-consuming and boring task to find each Ground Control Point (GCP) in 6-10 images and the chance of misidentification are high.

Some pieces of commercial software have an automatic GCP recognition system based on image processing algorithms, but no similar solution can be found in the open-source UAV image processing world.

I met ArUco codes some years ago and I used these black and white rectangular markers for indoor navigation and automatic movement detection. When I started to use ODM I realized the same technology can be used to find image coordinates of GCPs. There is a contrib package in OpenCV open-source library to detect marker positions and poses in images. The detection can be automatized using different ArUco markers for each GCP.

Different ArUco markers

At the Geo4All lab of the Department of Geodesy and Surveying at the Budapest University of Technology and Economics a small project was started to implement a solution in Python. The code and some instructions are available on GitHub Its usage is straightforward:

  1. Install Python3, OpenCV and OpenCV contrib on your machine
  2. Download the source code from GitHub (zip or clone)
  3. Generate ArUco markers (aruco_make.py or dict_gen_3x3.py) and print them in a suitable size
  4. Put the markers on the field before making UAV images and measure coordinates with the necessary accuracy
  5. Make the flight
  6. Collect the GCP’s 3D coordinates in a TXT file
  7. Run gcp_find.py
  8. Add the generated gcp_list.txt to your ODM project and start ODM processing
Original image and detected markers

Actually there are command line tools plus a simple visual check in the project. A GUI would be a useful addition to the project to edit and review GCP matching results possibly integrated in WebODM. More details are available on the GitHub page of the project and in an article in Baltic Journal of Modern Computing.

Contributions to this project are welcome, usage experience, case studies, bug reports and fixes, enhancements…

Ghost-busting, speed improvements and switch to AGPL with 2.3.0

 

Chapter 1. Ghosts

If there’s something strange, in your multispectral dataset, who you gonna call?

Well, the OpenDroneMap community obviously!

image
“Ghosts” due to multispectral band misalignment

Whenever ghosting occurs, the problem is related to band alignment. In multispectral datasets, sensors such as the Sentera 6x, take multiple pictures at once using different cameras, each capable of capturing a different part of the light spectrum. This enables us to have images for various bands such as red, green, blue and near infrared. The images however are not perfectly aligned, due to the relative position of the cameras and slightly different lens distortion.

Red, Green, Blue, NIR and Rededge band mis-alignment

ODM now includes state-of-the-art automatic band alignment using a hybrid approach of feature-based and Enhanced Correlation Coefficient (ECC) maximization alignment, which is capable of estimating the alignment parameters from a sample computed on a subset of the input images and applied to the entire reconstruction.

After alignment, things start to look much better!

Same bands, properly aligned

After aligning the bands, we simply make sure that our orthophoto / texturing generation process uses the sections of one band consistently with all the others, and the ghosts are officially… busted!

image
Before ODM 2.3.0
image
After ODM 2.3.0

If you’ve used OpenDroneMap in the past to process multispectral datasets and you’ve been less-than-enthusiast about the results… it’s time to give it another shot!

Chapter 2. Speed

We’ve also gotten a lot faster at processing these datasets too, courtesy of the awesome team at Mapillary which introduced more efficient multi-threading and a change in how we process multispectral datasets, which now require ~1/3 of the time compared to before. That’s a lot faster!

ODM is now also capable of generating Entwine Point Cloud (EPT) point clouds more quickly and with less memory usage, thanks to the awesome team at Hobu and their new Untwine software.

Chapter 3. AGPL License Switch

Starting from ODM 2.3.0, NodeODM 2.1.0 and WebODM 1.6.0, all three projects join the family of free software projects (which already included ClusterODM) that use the AGPL license.

This move is designed to continue guaranteeing the software freedom of our users in a world of ever-increasing cloud services. Despite some questionable claims of the otherwise, by certain large organizations, AGPL is an excellent license for an open source project such as OpenDroneMap (as this humorous blog post by Drew DeVault explains). This choice will continue to help OpenDroneMap be as open and inclusive as it possibly can.

Meet the Open Mobile Mapping System project

 

Today we’d like to share an impressive open source UAV hardware and software project, possible thanks to the effort of Ryan Brazeal and supporters.

When we think of open source UAV hardware projects we usually think of small, classroom friendly, tiny drones. Well, this one is not a toy, but will give a run for the money to the most sophisticated and professional rigs.

OpenMMS “presents a single source of information that lists every nut, bolt, OEM sensor, and manufacturing step required to build a modern, tightly-integrated MMS [Mobile Mapping System] sensor. The project also includes a robust suite of novel open-source software applications that rigorously process the collected data and generate industry-standard geospatial deliverables”.

Now for the kicker: “The complete OpenMMS solution provides many of the same features and achieves the same (or better) accuracy specifications as comparable commercial systems. The hardware combines a lidar sensor, global navigation satellite system sensors, an inertial sensor, cameras, and computers, to create a tightly-integrated digital mapping tool. The software is easy to use, fast, supports multiple OS, and, when possible, leverages multi-core (CPU) and graphics card (GPU) processing.”.

While OpenMMS is an independent project, here at OpenDroneMap we’re fully committed to support other open source projects in UAV mapping. We have opened a community channel on our forum for OpenMMS that will provide a venue for users to talk and share ideas about the project.

We look forward to see what the OpenMMS team will bring up next.

ODM 2.0 released! See what’s new.

 

Today we’re excited to announce a new major release of ODM! What have we been working on? Well a lot of things. The two most important ones are the ones we hope you won’t notice, because they don’t affect functionality, but have been part of necessary “infrastructure” updates to make sure that ODM continues to work in the future. Namely:

  • We’ve upgraded the codebase from Python 2 to Python 3. Python 2 has been deprecated and will not receive updates past 2020. With Python 3 support, ODM can continue moving forward with the rest of the Python ecosystem.
  • We’ve upgraded the base OS target from Ubuntu Linux 16.04 to 18.04. 18.04 will continue to receive extended security maintenance updates until 2028 and compatibility with 18.04 has been frequently requested from our community.

If you’re using docker, these changes are (should be) transparent. If you’re running ODM natively, you will need to upgrade your base system to 18.04 before updating ODM (or continue using a 1.x release of ODM until you decide to upgrade).

Aside from these important under-the-hood upgrades, we couldn’t make an important release such as a 2.0 release without adding something new and shiny!

  • Image Geolocation Files allow you to specify/override the GPS location of your images without having to fiddle with tools such as exiftool. This is different than using Ground Control Point files. With these you simply specify the location of the camera centers instead of the location of points on the ground.
  • Image Masks allow you to mask out areas that you don’t want to include in your reconstruction. This is useful in many scenarios. For example, during cell tower inspections, photos typically include parts of the sky, which end up creating strange artifacts and negatively affect the reconstruction. By using masks, one can delineate areas to exclude from a reconstruction, thus obtaining a clean and more accurate result.
  • A new option, –feature-quality automatically adjusts the image sizes for feature extraction based on predetermined ratios instead of relying on the user input or making assumptions about the image size.
  • Static tiles that were previously computed in NodeODM for use in viewers such as Leaflet or OpenLayers can now be generated via the –tiles option directly in ODM.

Aside from the speed improvements of having updated Python, PDAL, numpy and many other libraries, we’ve specifically improved memory usage in split-merge to handle even larger DEMs, have improved DEMs compression and improved speed/network stability in ClusterODM/NodeODM.

We’ve fixed numerous bugs and increased overall stability. See the related PRs for geeky details (#1156 and #124). We also cleaned up the ODM repository from old, large binary objects that were inflating the git repository. A git clone previously took 1.1GB of downloads, now it takes only 12MB. If you forked ODM, read this announcement as it affects you.

Give ODM 2.0 a try! If you find any issues, please report them on GitHub.

A new era of open source funding? How we raised $5,099 for core work.

 

As part of our mission to bring awesome drone software, we focus intensively on sustainability (a.k.a. how to pay for development, bug fixes, new features, hosting, etc. for the benefit of all).

Open-source is a tricky type of software to fund. I will not cover the details, as other people have already done a great job of summarizing the problem.

I have a keen interest in finding solutions to the open-source funding problem. In particular, how do you give more voice and decision-making power to the community? How can a community better steer the direction of the project? What features are worth implementing right away? What has value? What is noise?

When I stumbled on the idea of quadratic payments I knew that something interesting was up. The core ideas that came out of the quadratic payments paper as applied to open-source are that:

  • It’s not economically advantageous for people to directly fund open-source software. A system could make it advantageous by combining each pledge with additional funds from a sponsor pool. The additional funds optimally compensate for the value a pledge benefits other users. Example: if Bob makes a contribution ($25) for something that could be valuable to Alice, Bob needs to be compensated for the additional benefit that Alice gets ($25). So Bob’s pledge needs to be worth $50. More pledges lead to more overall value for everyone involved, making it economically advantageous for both Bob and Alice to pledge.
  • Pledging money toward the implementation of a feature/bug is not just a mean of payment, it’s also a vote of preference. It helps answer the question of “what’s important to me as a user?”.

So a month ago we built a platform to collect pledges from users and opened three funds with different funding goals. As excitement built-up, we successfully met the funding goal for one of the three funds with $5,099 worth of pledges.

Where does the “sponsor pool” money come from? It’s from one of the organizations that currently develops and benefits commercially from OpenDroneMap software. While ~90% of the funds come directly from this sponsor pool, the remaining 10% is community driven.

If you think this is not a lot, you might be missing the most important value of the funding process: the voting aspect of it.

As maintainers of FOSS we receive hundreds of feature requests. Time is limited however, so how do you prioritize? Thumbs up on GitHub / forum polling is a “one-person one vote” system, which is not ideal. If you let a single organization sponsor the development of a feature, you get a “one-dollar one-vote” system, which gives too much power to the single organization.

Quadratic voting is a fairer system. After the funding period has ended, we know what the community values and we know what to prioritize.

Oh, and we raised some funds in the process, too! Win-win?

We hope this model can be studied, improved and replicated in other FOSS communities. It could be the start of a new era of sustainable funding.

https://github.com/uav4geo/quadratik

Introducing the UAV Mapping Arena

 

With the help of several members of the community, today we have launched a simple tool to visually compare outputs processed with several drone mapping packages (including ODM, of course): https://opendronemap.github.io/UAVArena/

We are pleased with how far ODM has come over the years and hopefully this tool will continue to help track our progress, at least as a measure of “visual appeal” (up next, accuracy comparisons?).

WebODM Now Available As a Native Windows Application

 

Long time ago, when WebODM was still in its infancy, Windows was still my (Piero’s) primary development platform. As such, the very first versions of WebODM used to run natively on Windows, without docker. But as time went on, with docker entering the development/release process, Windows compatibility was placed aside and focus was shifted on getting docker to work less painfully on Windows (spoiler, it’s still painful).

Despite knowing that in theory WebODM could work natively on Windows , I don’t think anyone attempted to prove it. We’ve added a lot of stuff to WebODM since its old Windows-first days. Dynamic tiling, GRASS engine, background worker processes, message brokers, plugin systems, Python environments… none of which take Windows as a first-class citizen.

Curiosity… would it still work?

Well, I’m happy to report that, after some tweaks, the answer is

Thank you Keebler Elves! | nightshade

Not exactly a trivial endeavor, but the result is quite seamless:

  1. Run the setup.
  2. That’s it, there’s no other steps. Just run the software.
image
image

Ok, admittedly this requires that you have a processing node running NodeODM somewhere else, as we still haven’t managed to compile/run the full processing pipeline natively on Windows, this is just the user interface, but it’s a really important step toward full native support for Windows (which we have long-term plans for).

If you are used to the docker experience of installing WebODM, you might be seriously impressed by the simplicity of this new option.

The engineering feat is also quiet impressive, because getting WebODM to run natively requires:

  1. A web server
  2. A message broker service
  3. Multiple worker/scheduler processes
  4. A database (with PostGIS)
  5. A single Python environment
  6. An installation of GRASS
  7. Lots of smaller Python/Node.JS dependencies to work correctly

In particular, issues of compatibility had to be solved between GRASS’s Python environment and WebODM’s virtual environment, as the two can run in the same process space, but Windows does not like that at all. Piggy-backing GRASS’s Python environment to run all of WebODM’s other functions turned out to be a good solution and removed the need for multiple Python environments.

The application setups all of these requirements (with careful consideration for security as well, where strong passwords need to be set to protect from untrusted local users), launches the necessary services in the proper order and wraps the application using Electron. Voila’!

The windows setup is freely available from https://webodm.net. This is a first release, so a few issues are expected. If you find them, please report them.