Fixing problems at the core

Aug 2020 - Jan 2022

It's not about building band-aids. It's about asking "why?" enough that you've dug deep and uncovered the root cause.

Understand the reasoning behind users' asks and find the hidden pattern across user needs to deliver what users didn't even know they wanted.

Team

Myself and a software engineering architect

Simple request

It started with a simple ask. I was a designer on the Analytics team at Tableau, and users wanted better, richer data tables. They wanted tables to be easier to build, customizable ordering of columns, more flexible sorting, and the ability to add conditional formatting and sparklines. Tableau's experience shines in creating visualizations of all types, but the experience for tables, as one of the most basic ways to view data, was lacking.

Foundational obstacle

If only simple asks always had simple solutions. Tableau began life as a PhD thesis. The core insight that built the domain-defining product was a formalism, internally called VizQL, that enabled combining the query-building and visualization-defining steps into a single drag-and-drop exploratory experience. Fields from the underlying data source are shown as "pills" that can be dropped onto encoding channels, defined as "shelves" in the Tableau interface. The fields in play and how they are encoded define the query that needs to be built. The query returns a single result set table that is then used to create the visualization using very clearly defined rules of what information creates headers and axes, and what information creates marks. Users loved skipping preliminary steps, such as building queries, the ability for deep insight without knowing SQL, and the playful, free-form experience that supported the wandering needed for the process of analysis.

The core challenge, though, was that many rich tables required multiple flexible queries at different levels of detail to be built. This was incongruent with Tableau's approach, and thus, it wasn't clear how to build it into the product in a robust way.

Creative but painful workarounds

The Tableau community is incredibly creative. A quick search of the forums showed many examples of users explaining how to build rich tables. The write-ups showed extensive, complex steps. Many viewed tables as a basic visualization, and the fact that the user base had to build complicated workarounds was frustrating.

Many of the workarounds involved using Tableau calcs to hard-code layout logic or delicately building dashboards to behave like single visualizations. These approaches produce extremely fragile dashboards that break easily and pull users out of the flow of analysis during creation, undermining Tableau's key mission.

Patterns

When I chatted with an experienced architect on the team, he connected the core problem of rich tables to the core problem of map layers. Users wanted the ability to add different visualization layers to maps. This challenge was identical to that of rich tables. It required multiple query result sets to produce the multiple layers users expected to be able to draw. We didn't have two distinct problems, but one core challenge.

Exploration

With this realization, we proceeded to fill whiteboard after whiteboard with sketches and ideas to address this fundamental issue. We pored over the original thesis that built VizQL and spoke with the author, Chris Stolte. We reached out to people from across Tableau, got feedback from users, and created many prototypes. An idea started to emerge. Could we build layout logic into the fundamental level of the formalism?

Concept

We created Blocks. It was an evolution of Tableau's formalism that incorporated layout information into the process of analysis. We assembled a team of engineers and built a functional prototype for a company-wide hackathon. This prototype won the hackathon and brought more attention to our proposal. We ran a user study and wrote a paper. This paper won the best paper in such and such.

However, when you build a platform used by many, you can't simply change how it works at the core. Instead, you need to balance tactical features you can ship today that move you in the direction of the long term vision you developed.

I worked with the teams to ship improvements for building tables, and laid the groundwork for the map layers feature. I also partnered with other teams to improve other scenarios where information at multiple levels of detail needs to be visualized.

Separately, I partnered with several developers to design, build and user test a functional prototype that details the longer term vision we have to expand the system.

More information: https://www.tableau.com/blog/creating-richer-tables-using-blocks

An early flow showing the ease of creating a table, and the power of adding layout to the formalism.

An early flow showing how level of detail gets inherited through the layout tree.

A demo illustrating the flow of analysis using the functional prototype we built

A comparison with building a complex rich table in Tableau and the ease of building it in Blocks

Demonstrating how blocks allows you to create different, more efficient layouts of your tables

Diagrams showing examples of the user-facing UI, the resulting visualization, and the underlying layout tree that the system would create to define queries and build out the visualization in our formalism.

With the foundation of Blocks, we envisioned several other scenarios that could address other challenges Tableau users had.

Reflection

Following the path of investigation led us turning a simple ask into a proposal of a fundamental shift for the product. While we built up great momentum for our concept, and found a lot of support and positive feedback, it was too large of a shift for that point in time. However, using the concepts from this work, we scoped down to short term approaches for improving rich tables, adding map layers, and exploring ways to support new types of visualizations in Tableau.