Unraveling the Data Puzzle: Accelerate Data Value with Azure’s Data API Builder

Discover the missing piece to unlock your data’s full potential

Mar 15, 2023

Introduction

You heavily invested in modernizing your data landscape.

Not only, you successfully incorporated recent data from various sources into your analytical landscape and channeled extensive exertion into standardizing and harmonizing everything into a flexible schema. All of this through an automated pipeline — with data quality built-in!

Moreover, you have collaborated with the business and created a business-oriented information model enabling people to comprehend the data they’re looking at. Let’s now fill up that data mart. Presto!

As you reach the end of your journey, the anticipation of uncorking an expensive bottle of champagne is imminent. You achieved the pinnacle of success in transforming data into tangible “business value.”

Hold on! It appears we overlooked an important aspect — while we successfully ingested and transformed the data, the product owner started to furiously nag us about exposing this data to “real consumers.”

“Without consumption, no value,” he sneers.

You’re about to proudly hit the PowerBI button on your desktop when your CDO suddenly joins the call and begins to speak excitedly about “data products” that have been incorporated into concrete “business services,” formalized through “data contracts” and ultimately made available as “genuine APIs.” — Another Gartner conference must have taken place recently.

The excitement quickly starts to fade. Silence fills the virtual room.

The data engineers are tracking the obsessive smirk and rapid eye movements of the PO and CDO, who must be frantically exchanging messages through a back channel. This doesn’t look good, … at all!

Finally, someone finds the courage to unmute her microphone and voices her thoughts.

“We are data experts! Crafting APIs is something for a different breed of techies. It’s complex — not SQL-complex but next-level shit — and requires real coding. It even involves some voodoo called REST or let alone GraphQL. Just create a feature for the integration team. It should be out and about in less than 6 months from now.”

Suddenly, the energy shifts. All attention is now focussed on the elephant, err… Architect in the room.

The Product Owner launches his torpedo. “How exactly does this multi-million new data architecture of yours plans to address this?” he asks.

The Architects’ face pales. The Teams-channel is abruptly filled with a jittery sound, unmistakably emanating from the Architect’s rapid slamming on his Cherry keyboard. Half of his head has disappeared from the screen.

Internal murmurs follow, groaning noises, and suddenly a dreadful scream — of joy.

Appearing to have planned this all from the start, the Architect rises from his desk, adjusts his ring light, and fills the room with an air of confidence that only he can exude.

“Waiting on the integration team is not part of the plan; by the way, these guys are old-school. Furthermore, no one is expected to build APIs from scratch, as we are data experts, not geeks. We will simply generate our APIs.” —

On a whim, the anxiety dissipates from the room and is replaced by a sense of love and wonder.

The CDO takes charge of the moment and brings it to a close. “Well then, it’s settled! We’ll simply generate the APIs. I’ll inform our internal clients to prepare themselves, as we will have this operational in no time.”

The Puzzle of Data Value

In the data world, companies are constantly seeking innovative ways to unlock the actual value of their data without the hassle.

“It’s not about optimizing most of the chain, but the entire process….”

However, with the right tools and mindset, you can make sense of the chaos and accelerate data value. The tedious task of building and maintaining Data APIs — with built-in security and other essentials- is often overlooked or severely underestimated.

We’ll now explore how Azure Data API Builder can help us find that elusive missing piece of the puzzle.

Introducing Azure Data API Builder

Azure Data API Builder is an open-source solution that tackles these tedious tasks head-on.

With Azure Data API builder, it is possible to:

Expose database objects via REST or GraphQL endpoints, enabling access to your data via modern techniques on any platform, language, and device.
The integrated and flexible policy engine, coupled with native support for standard functionalities such as pagination, filtering, projection, and sorting, enables the creation of CRUD backend services within minutes instead of hours or days.
This provides Data teams with an unprecedented level of efficiency boost and self-sufficiency.
Moreover, the solution works against Azure SQL Databases but also honors the traditional on-premise SQL Server stack if that journey to the cloud has yet to happen.

Setting Up Your Azure Data API Builder

To start using Azure Data API Builder, follow these simple steps:

Clone the Azure Data API Builder repository from GitHub.
Set up your environment and install the required dependencies.
Configure your database and authentication settings.
Run the application and start building custom APIs
Go back to step 1 and adequately document, industrialize, and automate the process.

For detailed instructions and guidance, visit the project’s GitHub repository or consult the following comprehensive write-up by Aaron Powell: (Link).

Real-World Use Cases

Automatically generating REST and GraphQL APIs on top of an analytical data landscape can enable a variety of real-world use cases, including:

Mobile applications: By providing access to data via REST or GraphQL endpoints, developers can build applications that leverage the organization’s data assets without needing custom backend services.
Third-party integrations: REST and GraphQL APIs provide a standardized way for third-party systems to access and use data. This enables organizations to integrate their data with other systems, services, and organizations within their ecosystem.
Data monetization: Organizations can use REST and GraphQL APIs to provide access to their data for a fee. This enables them to monetize their data assets and create new revenue streams.
(Near) Real-time analytics: REST and GraphQL APIs can provide access to real-time data for use in analytics and machine learning applications. This enables organizations to make data-driven decisions in real time.
Self-service analytics: By providing self-service access to data via REST or GraphQL APIs, organizations can empower business users to build their own reports and dashboards without needing to rely on IT for assistance.

Conclusion

Azure Data API Builder is a powerful tool to help you find the missing piece of your data puzzle. By easily integrating data from disparate sources, creating custom APIs without friction, and ensuring data security, you can unlock the true potential of your data and accelerate its value.

Say goodbye to those ancient flat file exports or unresolvable dependencies on half your organization. Now is the time to explore Azure Data API Builder and make your customers happy by building modern data products at warp speed.

We close off with the wise words of Davide Mauri:

“Let’s remember to keep simple things easy! No need to always having the ultimate abstraction. Let’s aim for simplicity. We can always make things more complex later. The other way around is more complex.”

When you ever start thinking that something is “too simple for you,” please remember: You are not Google, Amazon or LinkedIn.