01 Big, geospatial, and well-designed

There were a few challenges we experienced in working on the Adopt a Storm Drain project.

First, scale: both the existing Rails app and the under-development app had the same inability to handle the large amounts of data and complexity of relationships that was required from the final app.

Next, geospatial: the frontend of the app requires only the latitude and longitude of a given drain. But the much more complex reporting system on the backend needs to have a deep understanding of the boundaries of each of the relevant geographic region — city, county, watershed, and subwatershed — and the ability to determine which regions any given drain falls within. This logic didn't exist at all in either existing app.

Additionally, there were non-technical challenges we didn't even expect at the beginning of the project. We needed to provide an entirely new series of designs that felt both light and modern but also mirrored the existing print designs. And we had to bring our user experience knowledge to bear to re-work some significant user experience problems present in the live Rails app.

02 The elegant monolith

The first question we ask when we encounter any pre-existing technical complexity is: “Why?”

When we learned the in-progress app was split into two codebases, a Lumen API and a WordPress plugin, we asked this question: Why? The answer that came back was inconclusive, so we had space to talk about other options.

Given the richness of the user experiences afforded by this application, we opted to merge the backend and the frontend into a single Laravel application, drastically simplifying the tech stack and simplifying future maintenance. Additionally, we could add a robust test suite much more easily than if it were two separate applications.

We then were able to build the application interface out quickly, saving the majority of our problem-solving time and attention for the more complicated aspects of the project: handling the scale and geospatial problems.

03 Databases in space

Tighten has a good amount of organizational knowledge when it comes to databases, but complex geospatial calculations are still relatively rare among web applications. Thankfully, one of our programmers had written a blog post about the foundations of this technology, and from that point we were able to reach out to some friends in the open source world and get a solid understanding and foundation of how to build the tooling.

We built utilities to locate and compare drains with their cities, counties, watersheds, and subwatersheds, performing millions of complicated calculations to map latitude and longitude against the earth's curvature. This presented challenges and opportunities for performance optimization, tests with multiple angles of attack for our costliest queries, and simple interfaces for repeated import of the same data over and over.

04 Performance at peak drain

The app launched with close to a half million drains, and the primary interface users have to the app is to look at the drains on a map. However, it's not feasible on an app this size to pull that many results from a database, nor to place that many points on a JavaScript-driven map.

Instead, we constrained the viewport and zoom level of the map and performantly live-refreshed the drains on the map from the API whenever the boundaries changed. This made the app feel quick, but only loaded a tiny subset of drains at any given time. We also utilized an extensive amount of caching and modifications to Laravel’s Eloquent calls to reduce the memory footprint of the larger calls.

05 Complex reporting, made fast and easy

Adopt a Storm Drain’s partners — cities, counties, watersheds, and subwatersheds — participate in the program in order to keep trash and debris out of their watersheds. But how do they know that the program is effective?

We used the brand-new-at-the-time admin tool Laravel Nova to provide easy access to the basic backend data with custom permissions levels for regional admins. We then wrote custom Nova reports to aggregate, visualize, and organize the data in ways their partners needed it.

Nova allowed us to build out basic admin tooling with almost no work, freeing up time (and providing a powerful skeleton) for building the custom reports and analytics the client needed.

06 A smooth transition and drastically improved user experience

When we went live, we imported hundreds of thousands of drains, thousands of users and adoptions, and thousands of regions from multiple data sources into the Laravel-based app. Within hours we were accepting hundreds of new adoptions, each notifying us in Slack.

Instead of seeing ten drains at a time on the map, users now see all of the drains available within many city blocks of their provided location. Admins can log in and see cleaning statistics, adoption records, and track their progress through the post-adoption workflow — all from a single Laravel app.