The Badger Data Platform, UW—Madison’s enterprise institutional data ecosystem, comprises services and tools designed to advance the use of our data and data-informed decision-making. The Badger Data Platform simplifies data access. The platform:
- Provides a single location to find and query data.
- Empowers the user to connect data from multiple systems.
- Uses a unified security model.
Services in the Badger Data Platform
Benefits of the Badger Data Platform
As our institutional data grows in complexity and volume, it is essential to provide the structure and support necessary for managing and analyzing it effectively. Historically, data-related services were independent of each other and didn’t always connect.
The Badger Data Platform provides a centralized, scalable, and flexible means of managing UW–Madison’s data to give our data consumers a clearer understanding of institutional data.
Historical environment
- Each system had its own security model.
- Joining datasets across systems was a difficult and error-prone process.
- Data sources often disagreed.
- Replicating data was the default and often the only solution.
Badger Data Platform
- Uses a single security model, based on data governance domains.
- Provides a location to analyze data from multiple warehouses, cloud applications, and systems in a single query.
- Designed to grow with the university’s ever-changing data environment.
In the Badger Data Platform, users no longer have to set up multiple data connections and drivers on their computers. They won’t have to request and manage security roles across different data sources. Lastly, multiple queries against each data source do not need to be run and then copied to a local source to combine them.
How data moves through the Badger Data Platform
Data that needs to be transformed, snapshotted, or exceeds the source system’s capabilities to be consumed directly can be processed into the Badger Data Warehouse. Denodo provides a single place to connect to data so that users can easily find and query data from many sources, such as the Badger Data Warehouse. Tableau is one of the many ways data can be consumed in the platform and serves as a location for both official institutional dashboards and divisional analysis dashboards.
Badger Data Platform details
The Badger Data Platform consists of multiple services and tools designed to complement each other. Below is a high-level visual that describes the platform’s core components and how they connect or relate to each other. A full written description follows the image.
Data source layer. At the bottom of the diagram, we identify some examples of the institution’s many data sources and lines of business. These data sources may include enterprise transactional systems such as the Student Information System (SIS), cloud platforms like Salesforce, or structured but external databases or warehouses like Amazon Redshift or Google BigQuery.
Data processing layer. As you move up the diagram, the next layer shows the various ways data can be accessed or extracted from those systems. For less structured data, such as system logs, data streaming tools can process the data. For processing data from transactional systems, we may use enterprise ETL (Extract, Load, Transform) tools, and systems that are performant enough can be accessed directly without replication.
Data storage layer. The next layer focuses on replicated data that needs to be stored in either the Badger Data Warehouse or the Badger Data Lake (coming soon). These tools serve a common purpose, but they meet different needs. Data that can be structured is best stored in a database, such as a warehouse, for ease of querying and updating. Unstructured or less structured data, such as log files, can be held in lake objects.
Data access layer. The data access layer, Badger Data Connect, provides a common tool for users to connect to the many sources of data from the lower layers using their single sign-on credentials and a common security and governance model. This layer helps provide semantic meaning to data from disparate sources, allowing users to query multiple systems in a single location.
It’s important to note that the arrows in the diagram that feed into the data access layer may come from the Lake, Warehouse, or directly from a source. This gives us the flexibility to apply different services as the use cases and system capabilities allow.
Data tools layer. The very top of the diagram represents the many tools adopted by the university that consume data from the Platform. While enterprise tools such as APIs or official Tableau dashboards will rely on the Platform for data, end-users will also need to bring their own tools to connect to the same data, such as:
- Data visualization tools (Tableau Desktop, Power BI).
- SQL editors (DBeaver).
- Code and scripting tools (Python, R Studio).
- Browser-based Denodo marketplace
Regardless of the tools used to connect to the data access layer, users see the same views of data based on their security and get the same results.
Questions and support
If you have any issues or questions about the Badger Data Platform, please email us at badgerdata@wisc.edu.