Switching from Java to Go? In this first part of the series, I share my journey transitioning from Java to Go, including key differences, challenges, and lessons learned. From implicit interfaces to error handling, multiple return values, and no function overloading—discover what makes Go unique and how to navigate the switch effectively. Whether you're considering Go or already making the move, this guide will help you adapt to Go’s simplicity and efficiency.
This blog explores how real-time data pipelines can cut fraud losses by 60%, reduce compliance costs by 50%, and drive multi-million dollar savings. Learn how fintech CTOs can leverage modern data architectures for sub-millisecond transaction processing, AI-driven risk management, and scalable infrastructure to future-proof their financial operations.
This blog explores how low-latency inference, hardware acceleration, and agile real-time data pipelines drive faster, smarter decision-making. Learn how Meroxa empowers organizations with seamless data ingestion, real-time analytics, and scalable infrastructure to bridge the gap between edge computing and actionable insights—unlocking the full potential of AI-driven innovation.
This blog explores how enterprises can leverage LLMs to extract instant value from continuous data streams and how conversational analytics is reshaping business intelligence. From financial services to retail, AI-powered data workflows are driving faster, smarter decisions—unlocking efficiency, scalability, and a strategic edge in a data-driven world.
As we mark the 3-year anniversary of Conduit Platform, we reflect on the incredible journey of innovation, scalability, and real-time data movement. Over the past three years, Conduit has transformed the way developers and data professionals build pipelines, enabling seamless data integration across diverse sources.
In this blog, we’ll look back at our biggest milestones, customer success stories, and the groundbreaking features that set Conduit apart. From powering real-time analytics to AI-driven data processing, Conduit has continued to push the boundaries of what’s possible in modern data infrastructure.
The latest Conduit 0.13 release brings significant upgrades, focusing on developer experience, automation, and performance optimization. Key highlights include automated documentation synchronization for connectors, a powerful new CLI for seamless pipeline and connector management, and 5x improvements in output processing speed, reducing latency and boosting efficiency. Notably, this version also deprecates the built-in UI, reinforcing Conduit’s commitment to a CLI-driven workflow. With expanded CLI capabilities and automated documentation tools, developers can now manage data pipelines more efficiently than ever. Upgrade today to leverage these new features and maximize your real-time data processing capabilities! 🚀
The Conduit 0.13 release introduces `connector.yaml`, a powerful automation tool that ensures connector documentation stays up to date with the latest configuration changes. By centralizing connector metadata—such as parameters, descriptions, and validation rules—developers can seamlessly sync documentation across repositories and the official Conduit docs. With the `conn-sdk-cli` tool, updating documentation is as simple as running a command, eliminating manual updates and reducing errors. This release enhances the developer experience, improves documentation consistency, and streamlines real-time data pipeline management. Start using `connector.yaml` today and automate your documentation workflow! 🚀
The Conduit 0.13 release introduces a powerful new CLI that simplifies real-time data pipeline management. With intuitive commands, developers can configure, monitor, and run pipelines directly from the terminal—eliminating complex API calls and manual configurations. Optimized for speed and efficiency, the CLI enhances deployment, troubleshooting, and high-throughput data streaming. Upgrade to Conduit 0.13 and streamline your data workflows today! Try it now with `$ conduit --help`. 🚀
No More Stale Models: Master Continuous MLOps with Meroxa & Databricks
Keep your machine learning models fresh and accurate with real-time, automated MLOps. This blog explores how Meroxa’s Conduit Platform and Databricks enable continuous model retraining**, eliminating stale predictions and manual updates. Learn how to streamline data movement, real-time feature engineering, and automated ML workflows for peak performance. Whether for fraud detection, predictive maintenance, or personalized AI, discover how to scale MLOps efficiently. Stay ahead with always-on machine learning—no more stale models!
At Meroxa, we’re empowering developers with Conduit OSS, a tool that simplifies real-time data engineering. With our latest release of new connectors, integrating with popular platforms is seamless, accelerating development and delivering real-time data insights. Here's a look at what's available now and what's coming next!
In this blog, we dive into how Conduit transitioned from a flexible but slower architecture to a high-performance streaming engine. We explore the limitations of the old DAG-based design, its impact on ordering guarantees and backpressure, and why switching to a Worker-Task Model drastically improved throughput and efficiency. With real-world performance benchmarks showing a 5x increase in message throughput, this blog is a must-read for developers and data engineers looking to optimize their data pipelines. Learn how Conduit is setting a new standard for real-time data movement and how you can leverage these improvements today! 🚀
This blog explores DeepSeek's hybrid training methodology, combining Supervised Learning and Reinforcement Learning, and emphasizes the critical role of real-time data orchestration for efficient LLM training. By showcasing how Meroxa’s platform enables dynamic data ingestion, seamless feedback loops, and scalable feature engineering, the blog provides actionable insights for professionals designing high-performance, real-time AI systems.
This blog dives into the critical differences between batch and real-time data pipelines, exploring why businesses are shifting towards real-time solutions to stay competitive. It highlights the limitations of batch processing—such as stale data, inefficiencies, and scalability challenges—while showcasing the benefits of real-time pipelines, including always up-to-date insights, proactive decision-making, and enhanced customer experiences. The blog also demonstrates how Meroxa’s Conduit Platform simplifies real-time data integration with features like in-flight transformations, cloud-native scalability, and real-time observability, making it the ideal tool for modern data workflows. Perfect for developers, data engineers, and business leaders seeking to harness the power of real-time data.
Whether you’re building recommendation engines, fraud detection systems, or dynamic pricing models, real-time data powers smarter and faster decisions. For Databricks users, the platform already excels at advanced analytics and AI. Pair it with Meroxa’s seamless, cost-effective real-time data ingestion and transformation capabilities to unlock the full potential of real-time AI workflows. Together, Meroxa and Databricks simplify complex streaming architectures, enabling accurate insights and faster business outcomes without the usual headaches or costs.
Discover how CTOs can tackle fragmented data, scalability, compliance, and vendor lock-in challenges. This blog highlights how Meroxa’s Conduit Platform enables real-time data integration, optimized costs, and innovation-ready architectures to future-proof your enterprise.
This blog provides a comprehensive, beginner-friendly guide to building your first real-time data pipeline using Meroxa’s open-source Conduit platform. Designed to simplify real-time data integration, Conduit empowers developers to move data seamlessly between systems with minimal setup. The guide walks readers through the installation, initialization, pipeline configuration, and execution steps. Starting with a sample pipeline that streams generated data to a destination file, it showcases Conduit’s intuitive configuration process and highlights the flexibility to handle structured, scalable data in real time. By the end, readers will have a fully functioning pipeline ready to deliver actionable insights and the foundational knowledge to explore advanced configurations. Whether you're new to real-time data or looking for a reliable integration tool, this guide demonstrates how Conduit makes real-time pipelines accessible to everyone.
This blog provides a comprehensive, beginner-friendly guide to building your first real-time data pipeline using Meroxa’s open-source Conduit platform. Designed to simplify real-time data integration, Conduit empowers developers to move data seamlessly between systems with minimal setup. The guide walks readers through the installation, initialization, pipeline configuration, and execution steps. Starting with a sample pipeline that streams generated data to a destination file, it showcases Conduit’s intuitive configuration process and highlights the flexibility to handle structured, scalable data in real time. By the end, readers will have a fully functioning pipeline ready to deliver actionable insights and the foundational knowledge to explore advanced configurations. Whether you're new to real-time data or looking for a reliable integration tool, this guide demonstrates how Conduit makes real-time pipelines accessible to everyone.
This blog highlights how integrating Meroxa for real-time data ingestion and Databricks for scalable processing transforms AI/ML workflows. This solution reduces data latency to under 30 seconds, accelerates model training from 48 to 6 hours, and boosts prediction accuracy by 23%. With automated pipelines and end-to-end integration, businesses save time, scale efficiently, and achieve tangible outcomes like improved customer engagement and increased revenue from real-time insights.
Discover how Meroxa revolutionized real-time data integration in 2024 with powerful new connectors, including Amazon DynamoDB, Snowflake, Apache Kafka, and more, enabling seamless data movement across diverse systems. This blog highlights key updates to the Conduit Platform, such as pipeline recovery features in Conduit v0.12.0 and schema registry support in Conduit Operator v0.0.2, designed to enhance resilience and simplify Kubernetes deployments. We also dive into a compelling case study featuring a global hotel network that leveraged Meroxa’s platform to integrate customer data, boosting guest satisfaction by 30% and increasing revenue per available room by 20%. Whether you're building robust data pipelines, enabling real-time analytics, or driving AI-powered transformations, explore how Meroxa’s latest advancements are unlocking new possibilities for developers and data teams.
Traditional data warehouses, long the cornerstone of analytics, are increasingly ill-equipped to meet the demands of today’s real-time, dynamic data needs. Enter the Lakehouse: an innovative architecture that blends the flexibility of data lakes with the robust querying capabilities of warehouses, seamlessly handling structured and unstructured data for real-time analytics. This blog explores the shortcomings of legacy systems, the transformative benefits of Lakehouses, and how Meroxa simplifies the journey with its intuitive tools, real-time data pipelines, and cloud-native scalability. From leveraging cutting-edge technologies like Apache Iceberg and Delta Lake to providing expert migration support, Meroxa empowers organizations to unlock faster, more flexible insights without the complexity or high costs of traditional solutions.
Discover how Meroxa's Conduit Platform AI empowers businesses to seamlessly integrate real-time data into Databricks for faster, smarter analytics. Learn how our platform reduces engineering effort by 60%, accelerates insights by 40%, and ensures 99.9% uptime with near-zero latency. From financial transactions to IoT data, this blog explores how Meroxa’s AI-driven pipelines simplify data movement, enabling you to unlock the full potential of Databricks for real-time decision-making.
In the rapidly evolving world of AI, businesses are discovering that the future lies not in massive, general-purpose language models (LLMs) but in tiny, specialized models powered by real-time data streams. These domain-specific models offer dramatic cost savings, enhanced accuracy, and reduced hallucinations by continuously learning from live business data. From customer support to financial services and supply chain management, tiny models excel in delivering precise, actionable insights tailored to specific operations. Powered by platforms like Meroxa, which enables robust real-time data infrastructure, this approach bridges the gap between AI capabilities and business needs, providing a sustainable, efficient path to enterprise AI innovation.
Champagne Week embodies the creativity, dedication, and collaboration of the Meroxa team, showcasing innovative projects like enhanced documentation, real-time IoT processing, AI-powered summarization, and a collaborative demo that highlights our growth. Thank you to everyone who contributed to making this week a success—and to our users, who inspire us to keep raising the bar.
In this blog, discover how Meroxa empowers governments to overcome the challenges of IT modernization with real-time data integration, cost-effective solutions, and secure, scalable infrastructure. Learn how Meroxa helps transform aging systems into agile, efficient platforms that meet the growing demands of today’s tech-savvy constituents. Explore real-world success stories, actionable insights, and innovative strategies to deliver better services, reduce costs, and improve transparency—without sacrificing security or control. Whether you're managing federal programs, state services, or local initiatives, Meroxa enables you to reimagine government operations at the speed of innovation.
In today’s fast-paced AI landscape, relying on outdated, static datasets can lead to inaccurate models, costly retraining cycles, and delayed time-to-market for AI features. Real-time data pipelines offer a game-changing solution, enabling AI systems to stay accurate, scalable, and cost-effective by continuously learning from current conditions. With benefits like a 40% reduction in model hallucinations, faster deployment, and lower infrastructure costs, real-time data is essential for building reliable applications such as fraud detection, recommendation engines, and Customer 360 profiles. Discover how Meroxa’s platform empowers organizations to implement real-time data pipelines and unlock the full potential of their AI initiatives.
Our latest article highlights why Meroxa’s Conduit Platform is the game-changing solution for real-time data streaming, offering up to 90% faster data delivery and 80% lower latency compared to batch-based tools like Fivetran.
The Hotels Network (THN) partnered with Meroxa to streamline data flow between their sales and support teams, overcoming siloed data and complex architecture challenges. Using Meroxa’s Conduit Platform, THN achieved a unified, real-time pipeline from Salesforce to Redpanda, reducing operational costs by 30% and enhancing customer support capabilities. The solution’s scalability ensures THN can continue to grow and optimize operations seamlessly.
We’re excited to introduce the latest update to the **Conduit Operator**, now with built-in **schema registry support**. This new feature allows seamless data encoding and decoding, improving data compatibility across your pipelines. Whether you're managing multiple Conduit instances or scaling your data operations, schema registry integration ensures a smoother, more reliable experience for handling complex data flows.
The Conduit team has just released Conduit v0.12, and we're gearing up for the launch of Conduit v1 with a focus on making pipelines more resilient. One key feature of this release is pipeline recovery, designed to automatically restart pipelines that experience temporary errors like network interruptions or service downtime.
With configurable backoff settings, Conduit can efficiently handle retries, reducing the impact of transient issues. Learn more about this feature and how it ensures your pipelines are always up and running.
Learn how to stream data from MongoDB to ClickHouse in real-time using Meroxa Conduit. This step-by-step guide simplifies data integration for scalable analytics and real-time reporting, empowering you to unlock insights faster.
ClickHouse has established itself as a prominent database for analytical applications due to its technical advantages over competitors like Druid, Pinot, and StarRocks.
We made it, Conduit v0.11 is here! In this latest release, we’ve focused on adding schema support, enabling you to detect schema changes and retain type information end-end.
We are thrilled to introduce our latest offering, the Conduit Platform, which brings a host of new features and improvements designed to enhance your real-time data streaming experience, now powered by our robust Conduit open-source core. This transformation brings enhanced performance, scalability, and usability, coupled with access to over 100 connectors maintained by our dedicated open-source community. Here’s a closer look at what’s new and how it can benefit your data operations.
Conduit connector for Apache Flink, a powerful combination that significantly expands Flink’s capabilities. Apache Flink is renowned for its robust stream processing capabilities, while Conduit offers a lightweight and fast data streaming solution, simplifying the creation of connectors.
At Meroxa, we recently adopted ArgoCD as our go-to continuous delivery (CD) tool, allowing us to easily leverage the GitOps framework to deploy Kubernetes resources and services. We utilize ArgoCD to simplify the management of tenant instances of our platform, deployed within a Kubernetes cluster.
Explore the power of Conduit to create custom connectors tailored to your specific data integration needs. Learn how to use the Conduit SDK for enhanced data management and discover a world of possibilities in streamlining your data workflows. Start building your custom connector today!
Explore the new features and enhancements in Conduit version 0.10, designed to streamline your data integration processes. Discover how our latest update can help improve efficiency, security, and performance for your data operations. Upgrade today and transform how you manage data with Conduit 0.10
Discover the excitement of Hackweek! Dive into our latest blog post to explore innovative projects and creative breakthroughs from our most recent Hackweek. Learn how teams collaborate to turn bold ideas into reality, fostering a culture of innovation. Perfect for tech enthusiasts and creative thinkers alike!
Explore the capabilities of Meroxa's Conduit HTTP Connector, a robust tool designed to enhance data integration by facilitating seamless communication with any API endpoint. Perfect for developers and enterprises looking to streamline data workflows and maximize connectivity. Discover how our HTTP Connector can transform your data management strategy.
Discover the revolutionary Conduit 0.9 update, enhancing data processing with standalone processors and advanced capabilities for seamless manipulation and efficiency. Explore now.
Discover the new Snowflake Conduit Connector, your solution for real-time data management challenges in Snowflake. This comprehensive guide covers everything from setup to performance insights, equipping you to implement real-time upserts and deletes seamlessly. Enhance your Snowflake experience with this essential tool for advanced data operations.
Conduit 0.8 more than doubles single-pipeline performance.
Revamp your data pipelines with Conduit and Redpanda! Swap Kafka and Kafka Connect complexities for a swift, user-friendly options.
Visit conduit.io to download and learn how to use Conduit, the secure and efficient open-source data integration tool accredited by the DoD Iron Bank.
Explore batching in Conduit connectors for improved data pipeline performance. Understand how it boosts throughput and scalability.
Conduit 0.7 gets us one step closer to being a full functioning, feature rich alternative to Kafka Connect.
Learn how OpenAI's GPT-4 has helped to streamline data connector building for Meroxa, reducing development time and effort.
The Spire Maritime AIS source data integration is the first of its kind. It works natively with Meroxa's stream processing data platform.
Meroxa enables big data projects to evolve with business needs and address data volatility challenges.
Choosing between speed and compliance is hard. Meroxa eliminates implementation complexity while still offering complete control of your data.
Learn how Meroxa's data platform can improve your time to value and enhance your experience when working with Confluent.
Meroxa helps you liberate your company's data so you can avoid vendor lock-in.
Conduit 0.6, is full of features and bug fixes that will help developers as they operate Conduit in production environments.
Because of Meroxa's revolutionary code-first strategy, Meroxa is the only independent streaming integration provider that frees you from vendor lock-in.
Ingest, transform & stream data from your Oracle Database with Meroxa in a few lines of code.
Learn how to implement the Medallion Architecture using Meroxa to streamline analytics and make it easier to work with large amounts of data.
Conduit 0.5: We made an easy-to-configure Dead Letter Queues (DLQ) through HTTP & gRPC, extending health checking, & adding capabilities with Debezium records.
Learn how to use the Meroxa Platform along with the Turbine Framework to transform, enrich, orchestrate, & analyze data in real-time.
Learn how Meroxa used Intermediate Representation (IR) to improve stream processing application efficiencies.
With Meroxa, companies optimize Snowflake storage, compute, operational costs and make their developers much more productive.
Meroxa saved money by replacing Workato with a Turbine data app which allows you to quickly sync, persist, and transform data between data infrastructures.
Now Ruby developers can stream Meroxa on Ruby. Meroxa is stream processing application PaaS.
Conduit is a tool to help developers build streaming data pipelines between production data stores and messaging systems.
Real-time data sync, transformation & migration from PostgreSQL to MongoDB using Meroxa with Change Data Capture (CDC).
Visualizing the Turbine data app provides insight into the runtime details of the application’s components on the platform.
Learn how to efficiently pull data out of MongoDB in real-time using a Meroxa Turbine data stream processing app.
Combining Ruby’s simplicity and power with Turbine, Rubyists can now build data streaming apps with development workflows you know and love.
Use Meroxa Turbine to call multiple APIs in real-time to transform & enrich your data via Clearbit, Apollo and Hubspot
Stream data from PostgreSQL to Apache Kafka using four lines of code with change data capture.
We decided to build a performance benchmark for Conduit early, so we could determine how much it can handle and what it takes to break it.
Starting a new job can feel like drinking from a firehose for the first few weeks. Meroxa has engineered the onboarding process to help create a smooth transition.
With Meroxa’s newest feature, developers can now easily invite their teammates to their account to share resources and build data applications together.
Building and maintaining a CLI is a daunting task if you don’t have some guidance along the way. We identified some best practices to help you.
Connector Middleware improves the developer experience. You can utilize middleware provided by the SDK to enrich the functionality of connectors without reinventing the wheel.
Conduit is a tool that helps developers move data within their infrastructure to the places they’re needed.
Bringing continuous delivery to Kafka and streaming data apps with Apache Kafka Connector and Feature Branch Deploys.
We are taking an important step towards helping customers build data applications with support for Apache Kafka as a resource on Meroxa.
Data apps may undergo several code changes during the development lifecycle. Feature branches allow developers to branch off the main or production instance of the data app code without impacting production code.
The opportunity to delight someone using your tool can happen at any time. The open-source Conduit project team makes the user experience a top priority.
Real-Time Analytics Using the Kappa Architecture in ~20 Lines of Code with Turbine, Materialize, Spark, & S3.
When automated testing of JavaScript apps, it is key to verify the app state is changing as expected. By emulating the DOM events in automated tests, you get closer to mimicking the app’s behavior accurately.
Using Turbine to build a data app to help source new sales leads helps small sales teams scale.
What if you could easily access and use categorical data to detect dangerous anomalies? With Turbine and thatDot Novelty Detector, you can.
By using Meroxa’s Turbine SDK, you can simplify the data activation process by reducing the need to use multiple point solutions for transformation and reverse ETL with code.
Testing streaming systems and architectures can be difficult because you need to mock data and have an upstream system continuously push that mock data. Conduit has made it easier with a built in generator that creates fake data for streaming systems..
We faced challenges with Protobuf, so we began looking for a resolution... enter Buf!
Candidates are often intrigued by perks like unlimited time off and flexible hours. But do those perks actually come through once they land the job?
If you want to do more than encourage your employees to spend money at a Black-owned restaurant on Juneteenth, here are some impactful actions your company can make.
A data app is an application that uses real-time or near-real-time events to solve a problem.
Change Data Capture (CDC) is a general term for a mechanism that communicates not just the current state of some data in an upstream resource.
Cinco de Mayo is here, and though many people make reservations with friends to eat at their favorite taco spot, let’s take time to honor Mexican heritage.
Since launching last April, Meroxa has become the de-facto platform for creating real-time data pipelines for over 300 companies.
Learn how to send and continuously sync data to Algolia using Turbine. With Turbine, you can properly test, review, and build data integrations in a code-first way.
Use Turbine, Meroxa's stream processing application framework, to perform real-time e-commerce Order Data warehousing and alerting.
Turbine offers a code-first approach to building real-time data lake ingestion systems. Build, review, and test data products with a developer's mindset.
We’re excited to share the next chapter of Meroxa and what it means for software engineers to build, test and deploy data applications.
In this release, Conduit now has an official SDK that will allow developers to build connectors for any data store.
Since Conduit ships as a tiny single binary, it functions as a powerful tool that allows you to efficiently move data from one place to another.
Conduit is a tool to move data around and Heroku is an application platform.
The Conduit roadmap is meant to provide insight into the major bodies of work we want to achieve within any given release.
The world is trending towards more rapid delivery of goods and services. We use the term “Real-time” to mean that it happens as close to “now” as possible.
Conduit is an open-source project to make real-time data integration easier for developers and operators.
The Future of the Modern Data Stack looks excellent for data engineers. But where is the modern data stack for software engineers?
We’re open-sourcing Conduit, Meroxa’s data integration tool built to be flexible & extendible, and provide developer-friendly streaming data orchestration.
Today, we’re excited to announce the Self-Hosted Environments Beta.
Step-by-step instructions on how to obtain a Meroxa access token. The Meroxa access token is needed to authenticate to the Meroxa API programmatically.
Let’s discuss the use cases of CDC and look at the tools that help you add CDC into your architecture.
Microsoft SQL Server is a powerful, widely used relational database management system. Today, we’re releasing a beta version of our Microsoft SQL Server connector.
Change Data Capture (CDC) is an efficient and scalable model that simplifies the implementation of real-time systems.
With the Meroxa CLI and the Meroxa Dashboard, your pipelines are streaming, real-time, and up and running in minutes, not months.
When you build a data pipeline using Meroxa, your data is encrypted in transit and at rest. Today’s platform update adds a new layer of security to Meroxa.
MySQL, one of the most popular open-source databases for developers, is now in public beta as a source and destination for real-time data streams.
Postgres Triggers and Functions are powerful features that allow you to listen for DELETE operations that occur within a table and insert the deleted row in a separate archive table.
Learn how to expose PostgreSQL Remotely Using ngrok. This method allows you to quickly test and analyze the behavior of PostgreSQL with Meroxa.
Analyzing the changes that occur to PostgreSQL will give you insight into the current state of the data and allows you to dig into the changes of your database.
“Data is the new oil.” If data is the new oil, we wanted to power the refinery. The merox process, but for data.
