Conduit 0.4 is here!

See what's new

Introducing Visualized Turbine Applications

By  Sara Menefee

 8 Dec 2022

We are excited to announce—software developers can now visualize what is happening behind the scenes with their Turbine data apps deployed to the Meroxa Platform. The application visualization provides insight into runtime details of subcomponents the Meroxa Platform builds and configures based on the code written with Turbine. Including the directional flow of data.

We designed the Turbine Application Framework with developer experience in mind. There's no need to learn a proprietary DSL (domain-specific language). Software developers can use their choice of supported programming language to build, test, and deploy robust data apps to process data streams. All while coexisting alongside an existing ecosystem of apps and services.

The Meroxa platform simplifies deployment by abstracting away the complexity required to build out and configure the various subcomponents necessary to run the data app and scale it dynamically, on demand, on our serverless architecture. The visualization aims to make this transparent.

Turbine Application Details

The app visualization lives in the application details page for your Turbine data apps. Here’s what you can expect.

Overview

Source

A Source is a required Resource that contains the upstream data for the Turbine data app. There is a limitation of one Source per Turbine data app as we do not yet support multiple Sources. You may however include any number of APIs in your function to help enrich the data stream.

The Source node in the app visualization communicate the name, type, state, collection (e.g. table, collection, index, etc), and last updated timestamp.

Source details

Function

A Function contains the custom code you have written using the process method. This is where you can transform or enrich data with any number of APIs from third-party platforms and services. The app visualization communicates the name, state, and last updated timestamp for the Function.

Function details

Destination(s)

A Destination is a Resource where data will be sent from the Turbine data app downstream. You may leverage any number of Destinations, or none at all. It’s up to you.

Destination nodes in the app visualization communicate the name, type, state, collection (e.g. table, collection, index, etc), and last updated timestamp.

Destination details

Data Flow

Between each node, you should see an arrow pointing in the direction of where the data is going. An arrow may originate from a Source Resource to a Function or Destination Resource. Or from a Function to a Destination Resource. This will show you where your data is going directionally. Please note, we do not yet validate whether data is moving.

Viewing and Access

You can access the app visualization on the details page in the Meroxa Dashboard for the Turbine data app. This details page can be accessed directly through the dashboard or via a URL when using select commands in the Meroxa CLI.

Dashboard

Log in to your Meroxa account. Once authenticated, you should land on the Applications tab. This will list all Turbine data apps deployed to your account along with their state. Click on the application name of choice to view the details page. This is where your app visualization may be accessed.

CLI

Within the Turbine data app local project file, you may use meroxa app describe in the Meroxa CLI will output details about your Turbine data app. Provided with details about the Turbine data app subcomponents that exist, there will be a URL that can be used to access the visualization in the Meroxa Dashboard.

If you are working outside of the Turbine data app local project file, you can use meroxa app describe appname.

$ meroxa app describe

    UUID:   123ab456-c7d8-91e0-fghi-j12k34lm56n
   Name:   yourappname
Language:   javascript
Git SHA:   ab1234c567de8910f1234g567891011h12i13j0k
Created At:   2022-11-16 19:22:26 +0000 UTC
Updated At:   2022-11-16 19:22:26 +0000 UTC
    State:   running
Resources
	pgdb (jdbc-destination)
		UUID:   12c228be-523c-477b-b4b5-2d25f6d05e8a
		Type:   postgres
		State:   running
	pgdb (debezium-pg-source)
		UUID:   98z765yx-432w-109v-u8t7-6s54r3q21p0o
		Type:   postgres
		State:   running

Functions
	anonymize-ab1234c
		UUID:   1a234bc-d567-8910-ef12-3456gh78ij90
		State:   running

    ✨ To visualize your application, visit https://dashboard.meroxa.io/apps/<yourappname>/detail

Using meroxa app list will display a table of all Turbine data apps deployed to your Meroxa account. This will include a direct URL to the Applications list page in the Meroxa Dashboard.

$ meroxa app list

ID              NAME           LANGUAGE     STATE
====== ======================= ============ ==========
584           liveapp          javascript   running
2980           fooapp          golang       degraded
3095           barapp          python       running

✨ To visualize your applications, visit https://dashboard.meroxa.io/apps

Have questions or feedback?

We love hearing from our customers! If you have questions or feedback, please feel free to contact us directly at support@meroxa.com or by joining our Discord community server.

🚀 We can’t wait to see what you build!

         Meroxa, Turbine, Real-time