Project: Apache Heron (App Definition and Development / Streaming & Messaging)
File: compiling-code-organization.md

id: version-0.20.2-incubating-compiling-code-organization title: Code Organization sidebar_label: Code Organization original_id: compiling-code-organization <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> This document contains information about the Heron codebase intended primarily for developers who want to contribute to Heron. The Heron codebase lives on . If you're looking for documentation about developing topologies for a Heron cluster, see instead. The primary programming languages for Heron are C++, Java, and Python. C++ 11* is used for most of Heron's core components, including the , and . Java 11* is used primarily for Heron's [topology API](heron-topology-concepts), and . It is currently the only language in which topologies can be written. Instructions can be found in , while documentation for the Java API can be found . Please note that Heron topologies do not require Java 11 and can be written in Java 7 or later. Python 2* (specifically 2.7) is used primarily for Heron's and UI components such as and the . Build tool* Heron uses as its build tool. Information on setting up and using Bazel for Heron can be found in . Inter-component communication* Heron uses [Protocol Buffers](https://developers.google.com/protocol-buffers/?hl=en) for communication between components. Most `.proto` definition files can be found in . Cluster coordination* Heron relies heavily on ZooKeeper for cluster coordination for distributed deployment, be it for or for a [custom scheduler](extending-heron-scheduler) that you build. More information on ZooKeeper components in the codebase can be found in the [State Management](#state-management) section below. The contains a variety of utilities for each of Heron's languages, including useful constants, file utilities, networking interfaces, and more. Heron supports two cluster schedulers out of the box: and a [local

The Java code for each of those schedulers can be found in , while the underlying scheduler API can be found Info on custom schedulers can be found in [Implementing a Custom Scheduler](extending-heron-scheduler); info on the currently available schedulers can be found in [Deploying Heron on Aurora](schedulers-aurora-cluster) and [Local Deployment](schedulers-local). The parts of Heron's codebase related to are mostly contained in . There are ZooKeeper-facing interfaces for , , and that are used in a variety of Heron components. The C++ code for Heron's [Topology Manager](heron-architecture#topology-manager) is written in C++ can be found in . The C++ code for Heron's [Stream Manager](heron-architecture#stream-manager) can be found in . The Java code for [Heron instances](heron-architecture#heron-instance) can be found in . The Java code for Heron's [Metrics Manager](heron-architecture#metrics-manager) can be found in . If you'd like to implement your own custom metrics handler (known as a metrics sink), see . Heron's API for writing topologies is written in Java. The code for this API can be found in . Documentation for writing topologies can be found in [Building Topologies](topology-development-topology-api-java), while API documentation can be found . Heron enables you to run topologies in for debugging purposes. The Java API for simulator can be found in . Heron's codebase includes a wide variety of example built using Heron's topology API for Java. Those examples can be found in . Heron has a tool called `heron` that is used to both provide a CLI interface for and to perform much of the heavy lifting behind assembling physical topologies in your cluster. The Python code for `heron` can be found in . Sample configurations for different Heron schedulers config can be found in , config can be found . The Python code for the can be found in . The Tracker is a web server written in Python. It relies on the framework. You can add new HTTP routes to the Tracker in and corresponding handlers in the directory. The Python code for the can be found in . Like Heron Tracker, Heron UI is a web server written in Python that relies on the framework. You can add new HTTP routes to Heron UI in and corresponding handlers in the directory. The Python code for the can be found in . The HTTP handlers and web server are defined in while the HTML, JavaScript, CSS, and images for the web UI can be found in the directory. There are a wide variety of tests for Heron that are scattered throughout the codebase. For more info see .