Turbine Feature Branch Deploys

At Meroxa we have committed to delivering exceptional developer experiences. Today, we are excited to introduce feature branch deploys—a first step toward enabling continuous delivery for Turbine data applications on the Meroxa Platform.

Data applications may undergo several code changes by one or many developers throughout the development lifecycle. Using feature branches, contributing developers can effectively branch off the main or production instance of their data application code. This allows them to further develop and test changes without impacting production code.

Deploying from feature branches enables developers to test the outcomes of their code directly against production data—a crucial step before merging and deploying their code to the production instance of their data application.

Deploying from a feature branch

In the following examples, we will walk you through the steps necessary to deploy a Turbine data application from a feature branch.

Create a feature branch

First, create a feature branch and name it something descriptive. The name you chose for your feature branch will automatically append to the end of your application name when deployed to Meroxa. This will help identify test instances to production instances of your Turbine data applications.

$ git checkout -b "transform"
Switched to a new branch 'transform'

Once a feature branch is checked out, you are ready to launch your code editor and begin making changes. When using feature branch deploys to test, we recommend you carefully review your Turbine code to ensure the appropriate data resources are used for testing. We recommend swapping out any downstream production data resources with test resources. This will help prevent any unintended updates to production data.

🎈 Note: Testing resources must be created and configured on Meroxa to be accessible to your Turbine data app test instances.

Commit your changes

Next, commit your code to prepare for deployment. Be sure to look over your Turbine code before committing your changes. It is also good to ensure you’re on the correct branch, you can check this by running the following command:

$ git branch
  main
* transform

Once you’ve confirmed you’re in the correct branch, commit your code with the following commands:

$ git add .
$ git commit -m "Anonymize PII field"
[transform 1a1234b] Anonymize PII field
1 file changed, 1 insertion(+), 1 deletion(-)

Deploy

Once the code is committed, you’re ready to deploy. Simply run the following command:

$ meroxa app deploy
Checking for uncommitted changes...
  ✔ No uncommitted changes!
✔ Feature branch (transform) detected, setting app name to users-transform...
Preparing application "users-transform" (golang) for deployment...
  ✔ Application built!
✔ Can access your Turbine resources
  ✔ Application processes found. Creating application image...
  ✔ Platform source fetched
✔ Dockerfile created
  ⠋ Creating "/Users/local/path/users" in "turbine-users-transform.tar.gz"
  ✔ "turbine-users-transform.tar.gz" successfully created in "/Users/local/path/users"
  ✔ Source uploaded
  ✔ Removed "turbine-users-transform.tar.gz"
  ⠋Removing Dockerfile created for your application in /Users/local/path/users
  ✔ Dockerfile removed
  ✔ Successfully built Process image! ("UUID")
  ✔ Deploy complete!
  ✔ Application "users-transform" successfully created!
  
✨ To visualize your application visit <https://dashboard.meroxa.io/apps/UUID/detail>

There you have it! You’ve successfully deployed from a feature branch. You can now check any downstream testing resources to see the outcomes of the changes made.

Validation errors

To protect from unintentional updates to your production data, the Meroxa Platform automatically validates resource collections referenced in your code. Here are some validation errors you may encounter as well as steps on how to resolve them.

Duplicate records validation

All destination resource collections referenced in your code are checked against Turbine data app instances already running on the Meroxa Platform. If another Turbine data app uses the same destination resource collection, the deployment process will be flagged by our validation. This is intended to prevent accidental record duplication in downstream resources:

$ meroxa app deploy
Checking for uncommitted changes...    ✔ No uncommitted changes!
✔ Feature branch (transform) detected, setting app name to users-transform...
Preparing application "users-transform" (javascript) for deployment...
  ✔ Application built!
  x Resource availability check failed
Error: ⚠️ Application resource "pg_user" with collection "orders" cannot be used as a destination. It is also being used as a destination by another application "users".
    
Please modify your Turbine data application code. Then run `meroxa app deploy` again. To skip collection validation, run `meroxa app deploy --skip-collection-validation`.

Looping validation

If a data app references a source resource collection that is the same as the destination resource collection in the Turbine code, this will result in the deploy process failing with a resulting error. This validation prevents accidental looping effects within a single Turbine data app. This does not detect loops across multiple apps within an account.

$ meroxa app deploy
Checking for uncommitted changes...
  ✔ No uncommitted changes!
✔ Feature branch (transform) detected, setting app name to users-transform...
  Preparing application "users-transform" (javascript) for deployment...
  ✔ Application built!
  x Resource availability check failed
Error: ⚠️ Application resource "pg_users" with collection "orders" cannot be used as a destination. It is also the source.

Please modify your Turbine data application code. Then run `meroxa app deploy` again. To skip collection validation, run `meroxa app deploy --skip-collection-validation`.

Here is an example of how this may manifest in your code:

exports.App = class App {
  async run(turbine) {
    let source = await turbine.resources("pg_users");
    let records = await source.records("orders");
    let destination = await turbine.resources("pg_users");
    await destination.write(records, "orders");
  }
};

Skip collection validation

There are some cases where you would want to bypass the above validations and deploy the application. For these scenarios, you can run meroxa app deploy --skip-collection-validation.

Have questions or feedback?

If you have questions or feedback, reach out directly by joining our community or by writing to support@meroxa.com.

We can’t wait to see what you build! 🚀