Strong, useful, beautiful: Designing a new way of getting data
By the Data Integration Design Team
What is good design? In the data integration design team, we ask ourselves this question every day as we strive to create products that meet the needs and expectations of our customers. Design is not just about aesthetics or functionality, but about creating meaningful and relevant experiences for customers. Every design tells a story. It tells a story about people: what they want and what they need.
One of the oldest definitions of good design comes from the Roman architect Vitruvius, who stated that a well-designed structure should have three qualities: firmitas (strength), utilitas (utility), and venustas (beauty). As product designers, content designers, and UX researchers, we believe that good design should be strong, useful, and beautiful. A good design should balance all three, especially the core foundational flows that all customers will interact with, like getting data.
Building a strong foundation
Getting your data is probably the most basic and important function of any data engineering or analytics task. Without it, none of the wonderful things that customers do with our applications would be possible, because data is the fuel for those results. Whether the goal is to generate a report, move data from one point to another, or have an AI model with great accuracy, all of them require customers to get data.
A diverse team of UX designers, researchers, product managers, and engineers, we explored, challenged, and iterated on all the potential ways to create a formidable structure. Through our research, we learned that while our customers desire change, they don’t want to be inconvenienced or disrupted by it. Armed with this key insight we kicked off daily working sessions, whiteboard exercises, and brainstorming efforts.
One of the major collaboration wins was creating a modular experience that served up the information in a predictable, dynamic way, catering to any scenario. From Home to OneLake data hub, local sources, templates, new sources, upload files, blank table, and blank query and a revolving set of reusable components (global search bar, tiles with individual three-party vendors, ordered list with sources and related information, and more), customers can now easily find their data and access it in a few clicks.
Creating useful and functional design
A useful design is one that solves a problem, meets a need, or provides a benefit. For example, a search engine is a useful design because it helps users find information quickly and easily. When it comes to getting data, our customers said that getting data is “the most basic and important process” in their day-to-day work.
Unsurprisingly, getting data is rarely our customer’s final goal – instead, getting data was an end to a means. It didn’t matter if their goal is to generate a report, move data from one point to another, or have an AI model with amazing accuracy, all of these tasks require data to get started.
Based on conversations with our customers, we knew having an easy search experience would be extremely useful. The best way to ensure usability is to use the words data analysts and data engineers use in their daily work life. The modernized get data leaves behind the simple keyword filtering and poor returned search results. Customer can engage with a powerful cognitive search that searches across all modules.
Additional features like filtering and sorting offer our customers a useful and meaningful way to organize their data. For instance, in the OneLake data hub, the most important things for a user to immediately engage with are their “Recent data” and “Recommended data.”
We’ve also crafted for accessibility and adaptability. One must-have for web experiences is responsive layouts, which shows a prioritized snapshot depending on a customer’s display size. When a customer reduces their app by 40%, the new view adapts from showing all source columns to merely the filename and the three most essential filters (All data, My data, and Endorsed in your org).
Making beautiful design is the final step
Before a design can be beautiful, both strong and useful principles must already be aligned. It’s not enough for the software to be strong and useful. Beauty is what builds upon it all.
We also want our customers to feel joy and success. And to that end, we sought out to create a frictionless experience. One where a customer simply goes about their job without overthinking it. We wanted the design to be invisible, so that all potential distractions didn’t. Only a clear direct path to getting the data and bringing it into the platform.
To achieve that, we zeroed in on crafting quiet, minimalistic design. The goal was to remove clutter and potential confusion.
For instance, customers may notice – or better yet, they won’t notice – that the multi-step processes are cleanly demarked with quick instructional headlines and actionable tools. If a new customer needs guidance, they can expand a tooltip for information and for all the seasoned customers, they simply see a streamlined flow.
Designing for the future of data integration
As we design Microsoft’s data integration product experiences for a SaaS environment, with modern design sensibilities and consumer-inspired interaction models, our goal is to look into the future of data engineering products and see what they could be: strong, useful, beautiful.
That passion drives us to spend time with our data and our customers to understand what works, what hurts, and how our ideas might delight them. And what better place to start than tackling the most fundamental step of any data activity: getting your data.
We’d love to hear what you think and how you think we can continue to improve your experience. We encourage you to review our Get data into Data Factory documentation and quick starts. There, you’ll learn about key concepts, additional tips for beginner and pro users alike, and recommendations on how to unlock the full potential of Data Factory.
