Apache NiFi extension creation

overview this guide provides end to end guidance for creating extensions using apache nifi–based templates the goal is to serve as a central reference page for extension developers, cloud practice teams, architects, consultants, and data engineers who need to build, configure, deploy, and manage extensions using the extensions framework the guide consolidates onboarding information, apache nifi fundamentals, access requirements, development workflows, sample templates, deployment guidance, and learning resources intended audience this guide is intended for extension developers cloud practice teams data engineers solution architects consultants new team members onboarding to integrations the guide assumes basic familiarity with , , containerized applications, and extension development concepts what is apache nifi? apache nifi is an open source, low code/no code data flow management framework that enables the creation of data integration pipelines through a visual drag and drop interface it supports real time data ingestion batch processing data transformation data routing security and governance controls monitoring and observability integration with multiple systems and cloud environments in kyndryl , apache nifi is a core component of the extensions framework the apache nifi extension template enables workflows to integrate with nifi and use its data ingestion, streaming, transformation, and routing capabilities across services solution architecture the extension enablement approach does not focus on developing apache nifi itself instead, it enables teams to build extensions by using preconfigured apache nifi templates provided through the extensions framework the solution consists of apache nifi development environment extension framework nifi base templates producer and consumer templates extension packaging framework kubernetes deployment environment catalog and extension management services developers build workflows visually in apache nifi and package those workflows as deployable kyndryl bridge extensions apache nifi fundamentals before creating extensions, become familiar with the following apache nifi concepts concept description flowfile represents data and metadata moving through a flow processor performs ingestion, routing, transformation, or delivery tasks connection queues data between processors process group groups related workflow components controller services shared services used by processors provenance tracks data movement and processing history nifi canvas visual development workspace configure the apache nifi development environment step 1 create an extension repository create a repository from the apache nifi extension template repository to accelerate extension development template repositories provide standard project structure deployment configuration containerization artifacts extension framework integration step 2 import the repository import the template repository into the local development environment open the development workspace select import select projects from git clone the repository select the required branch complete the import process step 3 build the extension container configure dockerfile deployment configuration certificates environment variables runtime settings build and run the container locally to validate configuration and extension behavior before deployment step 4 access the nifi canvas after the container starts open the apache nifi web interface sign in using the configured credentials access the nifi canvas begin extension flow development the canvas serves as the primary visual development environment create extensions using apache nifi templates step 1 select a template review available templates, including base template producer template consumer template team specific implementation examples use the template that best matches the target integration scenario step 2 design the data flow perform the following steps to build the extension flow adding processors configuring connections setting properties defining routing rules configuring error handling use reusable process groups and controller services whenever possible step 3 configure external systems configure required integrations, such as rest apis databases cloud services message queues third party platforms step 4 test the flow validate the following connectivity data transformation error handling logging performance security controls export and package the extension perform the following steps to export and package the extension after the development is completed export the flow definition from the nifi canvas review configuration settings replace secrets with environment variable references store the flow definition within the extension repository commit and push source code changes rebuild the container image if necessary deploying the extension deploy to the development cluster open the catalog select add catalog item provide the required extension information create the catalog item create an extension instance perform the following steps to create an extension instance open my extensions select create extension locate the catalog item select create instance provide configuration parameters create the instance start the extension monitor and troubleshoot review the following extension logs runtime health flow status processor errors data provenance records deployment events developers should also review deployment, monitoring, logging, and troubleshooting guidance available through data foundation resources