Project: Numaflow (App Definition and Development / Streaming & Messaging)
File: CHANGELOG.md

Update manifests to v1.2.1 fix: height fixes to render pipeline view (#1721) Vedant Gupta Yashash H L Update manifests to v1.2.0 fix: summary bar overlay fix for plugin (#1710) chore(deps): bump ejs from 3.1.9 to 3.1.10 in /ui (#1711) Vedant Gupta Yashash H L dependabot[bot] Update manifests to v1.2.0-rc5 doc: add "nav" for fallback-sink (#1694) doc: reduce streaming (#1689) doc: Fallback Sink (#1691) chore(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 (#1692) doc: session doc (#1650) Derek Wang Vigith Maurice Yashash H L dependabot[bot] Update manifests to v1.2.0-rc4 fix: version downgrade for monaco-editor (#1673) feat: Fallback Sink (#1669) fix: routing fixes (#1671) feat: controller change for fallback sink (#1664) Enable cors for numaflow api (#1631) feat: expose controller leader election duration and renew opts (#1657) Ali Ibrahim Derek Wang Vedant Gupta Yashash H L Update manifests to v1.2.0-rc3 fix: add headers to custom sinkrequest (#1653) fix: pass headers to transfomer (#1651) fix: avoid publishing watermarks for duplicate messages. (#1649) fix: flaky TestDropOnFull (#1647) fix: Dedup not working for multi-partitioned edge (#1639) fix: readonly view (#1640) Naga Vedant Gupta Yashash H L Update manifests to v1.2.0-rc2 feat: read only view for UI (#1628) fix: race condition while publishing wm inside reduce (#1599) fix: bug in late message handling for sliding window (#1471) fix: numaflow package style fixes (#1622) doc: add new user to the list (#1623) fix: watermark progression during pods creation/deletion (#1619) fix: allow pipeline to start with redis isbsvc (Fixes: #1513) (#1567) fix: dedup in user defined source (#1613) chore(deps): bump express from 4.18.2 to 4.19.2 in /ui (#1609) Quentin FAIDIDE Vedant Gupta Yashash H L dependabot[bot] Update manifests to v1.2.0-rc1 fix: flaky e2e tests (#1590) chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /ui (#1595) fix: error message that cause Buttons overflow in the UI (#1591) feat: support headers for message (#1578) feat: numaflow package (#1579) chore(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /ui (#1573) feat: read Kafka header and propagate in the payload (#1565) fix(codegen): protobuf upgrade (#1558) chore(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 (#1556) Chore: go 1.21 and k8s 1.29 (#1555) feat: unaligned wal (#1511) fix: initialize inflightAcks channel to not nil channel (#1548) chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#1549) fix: break from retry loop when key is not found (#1535) feat: noop persistence store for reduce (#1532) fix(controller): vertex template metadata nil check (#1527) feat: add support for SASL SCRAM 256 and 512 for Kafka (#1518) fix: incorrect json schema for tls config (#1520) fix: podSpec incorrectly configured in case template exist (#1516) add separate server info file paths for services on client side (#1494) fix: idle manager refactor for multi partitions (#1512) fix: avoid panic when ctx is canceled (#1515) fix: unknown for ISB details in pipeline card (#1497) feat: configure standardResources via controller configmap (#1490) fix: add idle handler offset nil check (#1489) feat: terminate reduce vertex pods when pausing pipeline (#1481) fix(controller): incorrect cpu/mem resources calculation (#1477) feat: pipeline health status for UI (#1460) fix: bug where dashed line is treated as permalink + spelling/formatting (#1467) feat: improve dex server (#1440) fix: memory leak inside session windower (#1445) fix: GetDownstreamEdges is not cycle safe (#1447) chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#1448) fix: UI Filter by status for pipelines doesn't work as expected (#1444) fix: Kafka source reads duplicated

(#1438) feat: enhance autoscaling peeking logic (#1432) fix: server-secrets-init container restart (#1433) feat: update tcp client connections (#1429) feat: Session Window and Reduce Streaming (#1384) doc: idle source (#1426) feat: health status implementation (#1406) chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1424) fix: configmap const name (#1423) Abdullah Hadi Ali Ibrahim Antonino Fugazzotto Damien RAYMOND Derek Wang Dillen Padhiar Juanlu Yu Nishchith Shetty Sidhant Kohli Vedant Gupta Vigith Maurice Yashash H L akash khamkar dependabot[bot] Update manifests to v1.1.7 fix: initialize inflightAcks channel to not nil channel (#1548) chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#1549) Antonino Fugazzotto Derek Wang dependabot[bot] Update manifests to v1.1.6 fix(controller): vertex template metadata nil check (#1527) feat: add support for SASL SCRAM 256 and 512 for Kafka (#1518) fix panic inside reduce after getting sigterm fix: incorrect json schema for tls config (#1520) fix: podSpec incorrectly configured in case template exist (#1516) fix: unknown for ISB details in pipeline card (#1497) feat: configure standardResources via controller configmap (#1490) fix: add idle handler offset nil check (#1489) feat: terminate reduce vertex pods when pausing pipeline (#1481) Damien RAYMOND Derek Wang Dillen Padhiar Juanlu Yu Nishchith Shetty Vedant Gupta Yashash H L Update manifests to v1.1.5 fix(controller): incorrect cpu/mem resources calculation (#1477) Derek Wang Update manifests to v1.1.4 fix: bug in late message handling for sliding window (#1472) Derek Wang Yashash H L Update manifests to v1.1.3 fix: GetDownstreamEdges is not cycle safe (#1447) chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#1448) fix: UI Filter by status for pipelines doesn't work as expected (#1444) fix: Kafka source reads duplicated messages (#1438) feat: enhance autoscaling peeking logic (#1432) Derek Wang Juanlu Yu Nishchith Shetty akash khamkar dependabot[bot] Update manifests to v1.1.2 fix: server-secrets-init container restart (#1433) Derek Wang Vedant Gupta Update manifests to v1.1.1 chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1424) fix: configmap const name (#1423) Derek Wang dependabot[bot] Update manifests to v1.1.0 feat: local user support for Numaflow (#1416) fix: consider lastPublishedIdleWm when computed watermark is -1 (#1415) fix: access path for auth endpoints (#1403) feat: Generate Idle Watermark if the source is idling (#1385) fix: include dropped messages in source watermark calculation (#1404) chore(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 in /ui (#1400) fix: updated access path config for root path (#1397) feat: improve numaflow k8s events (#1393) fix: update numaflow-go version (#1387) fix: access path for api/v1 route (#1388) fix: dropped messages should not be considered for watermark propagation (#1386) chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 (#1383) refactor: move udf forwarder to the right dir (#1381) fix: add pipeline update validation checks (#1379) fix: disallow updating an existing isbsvc's persistence strategy (#1376) fix: add more checks to isbsvc validation (#1358) fix: non-ack failed offsets (#1370) feat: validate patched data for pipelines (#1349) Add Atlan into USERS.md (#1351) Unit tests UI (#1348) fix(SERVER): remove unknown filter (#1346) Upstreammain (#1345) fix: rc-4 bug bash bug fixes (#1343) fix(UI): rc-0.4 fixes (#1342) fix(SERVER): fix styles for ISB cards fix(SERVER): fix styles for ISB cards fix(SERVER): fix for pagination issue fix(SERVER): namespace inout filtering space alignment fix(SERVER): pipeline card style fix fix: block pipeline load post update (#1333) fix: full isb spec in edit (#1331) RC2.0 UI fixes (#1329) feat: container for generator vertices (#1321) fix: create isb should move to isb tab (#1323) fix: max

(#1319) feat: cache daemon client for each pipeline (#1276) fix: rc2 UI fixes (#1317) feat: added filtering based on number, status and health of pipelines (#1312) fix: user identity cookie max age (#1316) feat: add scopes to authorization (#1288) fix: logout fix (#1310) fix: Split cookie to meet the cookie length requirement (#1305) chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307) add tooltips (#1289) fix: fixed the timer not clearing issue (#1303) feat: k8s events filtering and cluster summary card fixes (#1297) fix: graph overflow with large height (#1301) doc: need metrics server (#1296) refactor: unified metrics names for forwarders (#1290) feat: added separate colors for sideInput and dynamic legend (#1292) feat: add tabs to display pipelines and isb services (#1293) fix: more sidebar testing (#1287) refactor: create interfaces for AuthN and AuthZ (#1286) fix: tests for utils (#1283) Bradley Behnke Chandan Kumar Darshan Simha Derek Wang Dillen Padhiar Juanlu Yu Keran Yang Madusudanan.B.N Shakira M Sidhant Kohli Vedant Gupta Vigith Maurice Yashash H L dependabot[bot] mshakira Update manifests to v1.0.0 fix: rc-4 bug bash bug fixes (#1343) fix(UI): rc-0.4 fixes (#1342) Darshan Simha Derek Wang mshakira Update manifests to v1.0.0-rc4 fix(SERVER): fix styles for ISB cards fix(SERVER): fix styles for ISB cards fix(SERVER): fix for pagination issue fix(SERVER): namespace inout filtering space alignment fix(SERVER): pipeline card style fix fix: block pipeline load post update (#1333) fix: full isb spec in edit (#1331) RC2.0 UI fixes (#1329) feat: container for generator vertices (#1321) fix: create isb should move to isb tab (#1323) fix: max lag (#1319) Bradley Behnke Darshan Simha Derek Wang Juanlu Yu Shakira M Vedant Gupta mshakira Update manifests to v1.0.0-rc3 feat: cache daemon client for each pipeline (#1276) fix: rc2 UI fixes (#1317) feat: added filtering based on number, status and health of pipelines (#1312) fix: user identity cookie max age (#1316) Darshan Simha Derek Wang Dillen Padhiar Juanlu Yu mshakira Update manifests to v1.0.0-rc2 feat: add scopes to authorization (#1288) fix: logout fix (#1310) fix: Split cookie to meet the cookie length requirement (#1305) chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307) add tooltips (#1289) fix: fixed the timer not clearing issue (#1303) feat: k8s events filtering and cluster summary card fixes (#1297) fix: graph overflow with large height (#1301) doc: need metrics server (#1296) refactor: unified metrics names for forwarders (#1290) feat: added separate colors for sideInput and dynamic legend (#1292) feat: add tabs to display pipelines and isb services (#1293) fix: more sidebar testing (#1287) refactor: create interfaces for AuthN and AuthZ (#1286) fix: tests for utils (#1283) Bradley Behnke Darshan Simha Derek Wang Juanlu Yu Keran Yang Sidhant Kohli Vedant Gupta Vigith Maurice dependabot[bot] mshakira Update manifests to v1.0.0-rc1 fix: incorrect image version for namespaced numaflow-server (#1282) fix: ISBCreate test and fetch mock setup (#1279) fix: update dex to work with basehref (#1278) feat: AuthN/AuthZ for Numaflow UI (#1234) fix(doc): hpa api version (#1274) fix: updating example.md (#1262) Feat/side input tests (#1257) chore(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 (#1268) fix(UI): pod selection fix (#1266) doc: Numaflow high level security (#1264) fix: fixed the ns-summary page to allow creation of pipeline when no (#1263) Unit tests graph page (#1250) feat: Add e2e test for map sideinput, Fixes #1192 (#1211) feat: get current status of ISB service (#1199) Summary view fixes (#1253) chore(deps): bump github.com/nats-io/nats-server/v2 from 2.9.19 to

(#1232) fix: updated div's with box and removed unwanted css (#1236) chore(deps): bump @babel/traverse from 7.23.0 to 7.23.2 in /ui (#1221) Update kafka.md (#1218) Update generator.md (#1217) Update map.md (#1219) feat: UI 1.0 CRUD (#1181) fix: get isbsvc kind apiversion (#1220) Update kafka.md (#1215) Update generator.md (#1214) Update overview.md (#1213) feat: add udsource python e2e (#1204) doc: roadmap (#1208) Namespace card status bar changes 0.11 (#1206) feat: added unit tests for PipelineCard component (#1205) pods component error fix (#1203) fix: updated image styles to a class (#1202) feat: Changed the status bar component to an icon based component (#1198) fix(SERVER): restructure pod details component (#1189) feat: updated the legend to a collapsible one on the top left (#1196) chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190) feat: API Delete ISBSVC validation (#1182) added BCubed to the user list (#1184) feat: add timeout for pausing pipeline. Fixes #992 (#1138) feat: Jetstream support for replica of 1 Fixes #944 (#1177) doc: minor clean up of JOIN doc (#1175) fix: incorrect side inputs watch logic (#1164) fix: not considered as back pressured when onFull is discardLatest (#1153) fix(SERVER): handle states when status is unknown (#1154) fix(SERVER): fix key warning (#1152) Bradley Behnke Caroline Dikibo Chandan Kumar Darshan Simha Darshan Simha U Dennis Sosa Derek Wang Dillen Padhiar Joel Millage Jorvaulx Juanlu Yu Kayla Nussbaum Shubham Dixit Sidhant Kohli TASNEEM KOUSHAR Vigith Maurice aruwanip bpcarey01 dependabot[bot] mshakira Update manifests to v0.11.0 Namespace card status bar changes 0.11 (#1206) feat: added unit tests for PipelineCard component (#1205) pods component error fix (#1203) fix: updated image styles to a class (#1202) feat: Changed the status bar component to an icon based component (#1198) fix(SERVER): restructure pod details component (#1189) feat: updated the legend to a collapsible one on the top left (#1196) chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190) added BCubed to the user list (#1184) feat: add timeout for pausing pipeline. Fixes #992 (#1138) feat: Jetstream support for replica of 1 Fixes #944 (#1177) doc: minor clean up of JOIN doc (#1175) fix: incorrect side inputs watch logic (#1164) Darshan Simha U Derek Wang Dillen Padhiar Joel Millage Vigith Maurice dependabot[bot] mshakira Update manifests to v0.11.0-rc2 fix: not considered as back pressured when onFull is discardLatest (#1153) fix(SERVER): handle states when status is unknown (#1154) fix(SERVER): fix key warning (#1152) Derek Wang mshakira Update manifests to v0.11.0-rc1 feat: Numaflow UI 1.0 (#1077) fix: treat ALL user-defined source vertices as scalable (#1132) feat: add doc link checker (#1130) extracting redis streams source (#1113) refactor: shared kubeconfig util (#1095) fix: wrong api group in webhook rbac settings (#1086) feat: set correct number of replices for some types of vertices after resuming pipeline (#1085) chore(deps): bump graphql from 16.6.0 to 16.8.1 in /ui (#1078) feat: add forest validation for pipelines. Fixes #1002 (#1063) feat: implement pipeline validation for unsupported states (#1043) fix: message count read in forwarder (#1030) fix(ci): Wrong test pod image tag used for running CI on release branch (#1041) fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040) refactor: re-arrange some of the rater implementations (#1036) fix(docs): file names and links for side inputs (#1037) doc: Add side input docs (#1029) doc: kill-switch when buffer is full (#1034) fix: Idle handler refactor (#1021) fix: calculate processing rate for sink

(#1025) feat: colored logs for UI with toggle for logs order (#1022) Derek Wang Dillen Padhiar Juanlu Yu Julie Vogelman Keran Yang Sidhant Kohli Tianchu Zhao Vedant Gupta Vigith Maurice dependabot[bot] Update manifests to v0.10.1 feat: implement pipeline validation for unsupported states (#1043) fix: message count read in forwarder (#1030) fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040) refactor: re-arrange some of the rater implementations (#1036) fix(docs): file names and links for side inputs (#1037) doc: Add side input docs (#1029) doc: kill-switch when buffer is full (#1034) fix: Idle handler refactor (#1021) fix: calculate processing rate for sink vertices (#1025) feat: colored logs for UI with toggle for logs order (#1022) fix(ci): Wrong test pod image tag used for running CI on release branch (#1041) Derek Wang Dillen Padhiar Juanlu Yu Keran Yang Sidhant Kohli Vedant Gupta Vigith Maurice Update manifests to v0.10.0 fix: seg fault inside controller (#1016) fix: reconcile headless services before pods (#1014) fix: print version info when starting (#1013) feat: join vertex UI support (#1010) feat: scaleUpCooldownSeconds and scaleDownCooldownSeconds to replace cooldownSeconds (#1008) chore(deps): bump @adobe/css-tools from 4.2.0 to 4.3.1 in /ui (#1005) fix: avoid unwanted watcher creation and reduce being stuck with udf is restarted (#999) fix: missing edges on UI (#998) feat: Add side input sdkclient and grpc (#953) feat: implement user-defined source (#980) fix: send keys for udsink (#979) fix bulleted list (#977) refactor: build wmstore and wmstorewatcher directly, and remove some unnecessary fields (#970) feat: add vertex template to pipeline spec (#947) feat: Add side-input initializer and synchronizer (#912) fix: npe when the ctx is canceled inside kv watcher (#942) fix: retry logic for fetching last updated kv time (#939) fix: close the watermark fetcher and publishers after all the forwarders exit (#921) Pipelines with Cycles: e2e testing, and pipeline validation (#920) docs quick fixes (#919) docs updates (#917) feat: watermark delay in tooltip (#910) fix: removing WIP tag (#914) feat: emit k8s events for controller messages. Fixes #856 (#901) fix: avoid potential deadlocks when operating UniqueStringList (#905) refactor: avoid exposing internal data structures of pod tracker to the rater (#902) feat: Join Vertex (#875) fix: stabilize nats connection (#889) doc: Update multi partition doc (#898) fix: Reduce idle WM unit test fix (#897) updated default version of Redis used for e2e (#891) fix TestBuiltinEventTimeExtractor (#885) chore(deps): bump word-wrap from 1.2.3 to 1.2.4 in /ui (#881) fix: remove retry when the processor is not found. (#868) refactor: create a new data forwarder dedicated for source (#874) feat: controller changes for Side Inputs support (#866) fix: highlight edge when buffer is full (#869) fix: minor ui bugs (#861) fix: release script for validating webhook (#860) fix: use windower to fetch next window yet to be closed (#850) feat: implement optional validation webhook. Fixes #817. (#832) chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845) Derek Wang Dillen Padhiar Jason Zesheng Chen Juanlu Yu Julie Vogelman Keran Yang RohanAshar Sidhant Kohli Vedant Gupta Vigith Maurice Yashash H L dependabot[bot] Update manifests to v0.9.3 fix: seg fault inside controller (#1016) Derek Wang Yashash H L Update manifests to v0.9.2 fix: error when kv_watch with no keys (#981) Derek Wang Update manifests to v0.9.1 fix: npe when the ctx is canceled inside kv watcher (#942) feat: watermark delay in tooltip (#910) Derek Wang Vedant Gupta Yashash H L Update manifests to

feat: emit k8s events for controller messages. Fixes #856 (#901) fix: avoid potential deadlocks when operating UniqueStringList (#905) fix: stabilize nats connection (#889) fix: Reduce idle WM unit test fix (#897) fix TestBuiltinEventTimeExtractor (#885) fix: remove retry when the processor is not found. (#868) fix: highlight edge when buffer is full (#869) fix: minor ui bugs (#861) fix: release script for validating webhook (#860) Derek Wang Dillen Padhiar Juanlu Yu Keran Yang Vedant Gupta Yashash H L Update manifests to v0.9.0-rc2 fix: use windower to fetch next window yet to be closed (#850) feat: implement optional validation webhook. Fixes #817. (#832) chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845) Derek Wang Dillen Padhiar Yashash H L dependabot[bot] Update manifests to v0.9.0-rc1 fix: pod tracker logic for calculating processing rate (#838) chore(deps): bump tough-cookie from 4.1.2 to 4.1.3 in /ui (#839) fix: resource leak inside daemon server (#837) feat: capability to increase max message size (#835) doc: update roadmap (#830) feat: add stragglers (late data) into the window is window is open (#824) fix(docs): fixed some incorrect docs and renamed a timeExtractionFilter arg (#814) feat: rater changes to track processing rate per partition (#805) fix: metric to track watermark bug was wrongly tagged (#809) feat: autoscaling changes to support multi partition (#806) fix: segmentation fault in daemon server (#804) fix: Intermittent failure from Kafka to get consumer offsets (#803) feat: support UI for multipartition edges (#789) refactor: remove redundant delta calculations for rater (#795) fix: select pods not in evicted status (#786) feat: combine built-in UDTransformers for filter and eventTime assignment (#783) feat: support multi-partitioned edges (#751) fix: duplicate ui served from gin Router (#781) fix: unexpected high processing rates (#780) chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#772) feat: gRPC error handling (#744) feat: forwardAChunk to support multi partitioned edges (#757) fix: pipeline view fix (#755) fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756) feat: use metrics to calculate vertex processing rate (#743) feat: enable streaming message to next vertex when batch size is 1 (#709) fix: use int32 for message length (#750) feat: using one bucket for partitioned reduce watermark propagation (#742) fix(test): flakey test (#738) refactor: buffer, edge, bucket (#733) feat: change baseHref for Numaflow UI. Fixes #375. (#698) fix: let kafka source crash and restart when there is any server side error (#735) feat: Autoscale for Redis Streams Source (#726) feat: Redis7 as an ISB svc (#717) chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724) docs(proposal): edges, buffers and buckets (#704) chore(doc): update README with demo (#718) doc: add overview (#713) feat: allowedLateness to support late data ingestion (#703) fix: allow late message as long as window is not closed (#696) fix: add wal dir x permission (#689) chore(doc): refactor doc struct (#685) Derek Wang Dillen Padhiar Juanlu Yu Julie Vogelman Keran Yang Vedant Gupta Vigith Maurice Yashash H L dependabot[bot] xdevxy Update manifests to v0.8.1 fix: pipeline view fix (#755) fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756) feat: use metrics to calculate vertex processing rate (#743) feat: enable streaming message to next vertex when batch size is 1 (#709) fix: use int32 for message length (#750) feat: using one bucket for partitioned reduce watermark propagation (#742) fix(test): flakey test (#738) refactor: buffer, edge, bucket (#733) feat: change baseHref for Numaflow UI. Fixes

(#698) fix: let kafka source crash and restart when there is any server side error (#735) feat: Autoscale for Redis Streams Source (#726) feat: Redis7 as an ISB svc (#717) chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724) Derek Wang Dillen Padhiar Julie Vogelman Keran Yang Vedant Gupta Yashash H L dependabot[bot] xdevxy Update manifests to v0.8.0 feat: allowedLateness to support late data ingestion (#703) fix: allow late message as long as window is not closed (#696) fix: add wal dir x permission (#689) chore(doc): refactor doc struct (#685) Derek Wang Vigith Maurice Update manifests to v0.8.0-rc1 feat: introducing tags for conditional forwarding (#668) feat: expose cpu/mem info to sidecar containers (#678) feat: Redis Streams source fixes (#669) fix: skip empty Kafka partitions when calculating pending count (#666) feat: support for multi keys (#658) feat: Adds SASL (plain and gssapi) support for kafka sink (#656) fix: vertex overlapping watermark (#660) feat: incremental search and namespace preview in search bar (#654) feat: integrate serde WAL (#650) fix: unit test (#653) feat: handle idle watermark for reduce vertex (#627) feat: Redis streams source (#628) feat: Adds SASL (plain and gssapi) support for kafka source (#643) feat: namespace scope api and disable namespace search on UI (#638) fix: GetHeadWatermark Logic (#636) feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634) fix: IdleWatermark unit test (#640) fix: desired replicas should not be greater than pending (#639) fix: add timeout to the test (#618) feat: kustomize integration (#637) fix: exclude ack pending messages (#631) feat: UI error component (#613) fix: do not update status.replicas until pod operation succeeds (#620) feat: track and expose oldest work yet to be done to the reduce loop (#617) feat: handle watermark barrier for map vertex (#607) fix: corrected reduce vertex replica number. Fixes #593 (#616) feat: add API for pipeline status check. Fixes #407. (#599) chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610) feat: use randomized shuffle using vertex name as the seed (#601) fix: ack the dropped messages as well (#603) feat: enable controller HA (#602) feat: expose dnspolicy and dnsconfig to pod template (#598) Chore: tickgen changes to test reduce pipelines (#587) feat: bidirectional streaming (#553) feat: use customized binary serde for nats message payload (#585) fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575) feat: implement watermark propagation for source data transformer (#557) feat: namespace search (#559) fix: refine log for buffer validation. Fixes #185 (#573) feat: add readiness and liveness check for daemon server. Fixes #543 (#571) feat: marshal/unmarshal binary for read message (#565) chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568) chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567) feat: expose image pull policy to user defined containers (#563) fix: typos in reduce examples (#556) feat: edge-watermark (#537) feat: enable envFrom for user defined containers (#554) feat: remove secret watch privilege dependency (#542) fix: Use a copied object to update (#541) chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539) feat: improve reduce performance (#501) feat: Offset time idle watermark put (#529) fix: securityContext not applied to container templates (#528) feat: idle watermark v0 (#520) feat: Reduce UI Support (#500) feat: enable RuntimeClassName for vertex pod (#519) feat: add builtin filter and event time extractor for source transformer (#517) chore(deps): bump ua-parser-js from

to 0.7.33 in /ui (#507) Add an e2e test for source data transformer (#505) feat: Implement source data transformer and apply to all existing sources (#487) fix: -ve metrics and return early if isLate (#495) fix: JetStream context KV store/watch fix (#460) doc: reduce persistent store (#458) doc: reduce documentation (#448) chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454) refactor: simplify http request construction in test cases (#444) refactor: use exact matching instead of regex to perform e2e data validation. (#443) doc: windowing fixed and sliding (#439) refactor: move redis sink resources creation to E2ESuite (#437) refactor: a prototype for enhancing E2E test framework (#424) feat: pipeline watermark (#416) Alex Ang HH Derek Wang Dillen Padhiar Juanlu Yu Julie Vogelman Keran Yang Vedant Gupta Vigith Maurice Yashash H L ashwinidulams dependabot[bot] Update manifests to v0.7.3 feat: integrate serde WAL (#650) fix: unit test (#653) feat: handle idle watermark for reduce vertex (#627) feat: Redis streams source (#628) feat: Adds SASL (plain and gssapi) support for kafka source (#643) feat: namespace scope api and disable namespace search on UI (#638) fix: GetHeadWatermark Logic (#636) feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634) fix: IdleWatermark unit test (#640) fix: desired replicas should not be greater than pending (#639) fix: add timeout to the test (#618) feat: kustomize integration (#637) fix: exclude ack pending messages (#631) feat: UI error component (#613) fix: do not update status.replicas until pod operation succeeds (#620) feat: track and expose oldest work yet to be done to the reduce loop (#617) feat: handle watermark barrier for map vertex (#607) fix: corrected reduce vertex replica number. Fixes #593 (#616) feat: add API for pipeline status check. Fixes #407. (#599) chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610) Alex Ang HH Derek Wang Dillen Padhiar Juanlu Yu Julie Vogelman Keran Yang Vedant Gupta Vigith Maurice Yashash H L dependabot[bot] Update manifests to v0.7.2 feat: use randomized shuffle using vertex name as the seed (#601) fix: ack the dropped messages as well (#603) feat: enable controller HA (#602) feat: expose dnspolicy and dnsconfig to pod template (#598) Chore: tickgen changes to test reduce pipelines (#587) feat: bidirectional streaming (#553) feat: use customized binary serde for nats message payload (#585) fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575) feat: implement watermark propagation for source data transformer (#557) feat: namespace search (#559) fix: refine log for buffer validation. Fixes #185 (#573) feat: add readiness and liveness check for daemon server. Fixes #543 (#571) feat: marshal/unmarshal binary for read message (#565) chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568) chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567) feat: expose image pull policy to user defined containers (#563) fix: typos in reduce examples (#556) feat: edge-watermark (#537) feat: enable envFrom for user defined containers (#554) Derek Wang Dillen Padhiar Juanlu Yu Julie Vogelman Keran Yang Vedant Gupta Vigith Maurice Yashash H L ashwinidulams dependabot[bot] Update manifests to v0.7.1 feat: remove secret watch privilege dependency (#542) fix: Use a copied object to update (#541) chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539) feat: improve reduce performance (#501) feat: Offset time idle watermark put (#529) fix: securityContext not applied to container templates (#528) feat: idle watermark

(#520) feat: Reduce UI Support (#500) feat: enable RuntimeClassName for vertex pod (#519) feat: add builtin filter and event time extractor for source transformer (#517) chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /ui (#507) Add an e2e test for source data transformer (#505) feat: Implement source data transformer and apply to all existing sources (#487) fix: -ve metrics and return early if isLate (#495) Derek Wang Juanlu Yu Keran Yang Vedant Gupta Vigith Maurice ashwinidulams dependabot[bot] Update manifests to v0.7.0 fix: JetStream context KV store/watch fix (#460) doc: reduce persistent store (#458) doc: reduce documentation (#448) chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454) refactor: simplify http request construction in test cases (#444) refactor: use exact matching instead of regex to perform e2e data validation. (#443) doc: windowing fixed and sliding (#439) refactor: move redis sink resources creation to E2ESuite (#437) refactor: a prototype for enhancing E2E test framework (#424) feat: pipeline watermark (#416) Derek Wang Juanlu Yu Keran Yang Vedant Gupta Vigith Maurice dependabot[bot] Update manifests to v0.7.0-rc1 feat: reduce metrics. Closes #313 (#414) feat: udsink grpc stream (#421) chore(doc): scope UDF under a dir (#426) feat: sliding window. closes #339 (#354) refactor: nats/jetstream testing (#418) feat: nats as source (#411) fix: adding lock while discovering partitions, Closes #412 (#413) fix(test): e2e-api-pod can not start on M1 mac (#410) fix: getWatermark to return-1 if any processor returns -1 (#402) fix: e2e testing for PBQ WAL with reduce pipeline (#393) feat: add Grafana instruction and a dashboard template. Closes #287 (#381) fix: unit tests for replay. Closes #373 (#377) chore(docs): update docs (#380) fix: best effort processing during SIGTERM. Closes #371 (#372) feat(wal): First pass to implement WAL and hook to PBQ store. (#344) feat: watermark otwatcher enhancement (#364) refactor(docs): group docs in categories (#362) chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356) fix(controller): vertex nil check for edge listing. Fixes #352 (#353) fix: data race in pbq manager. Closes #348 (#349) Chore: Windower interface. closes #234 (#340) feat: add minikube, kind and podman support (#206) refactor: Close watermark fetcher and publisher correctly (#336) passing window information inside the context (#341) feat: timestamp in UI to display milliseconds. closes #280 (#337) Simple reduce pipeline. Fixes #289 (#317) feat: add blackhole sink. Closes #329 (#330) fix: move watermark based on the head of the read batch (#332) feat: configurable jetstream storage (#328) feat: support adding sidecars in vertex pods. Closes #323 (#325) feat: populate watermark settings to vertex spec. Closes #320 (#321) doc: add few use cases (#318) Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315) fix: jetstream build watermark progressors bug (#316) feat: update watermark offset bucket implementation (#307) feat: shuffling support (#306) feat: customize init-container resources. Closes #303 (#304) feat: watermark - remove non-share OT bucket option (#302) feat: customization for batch jobs. Closes #259 (#300) refactor: abstract pod template (#296) feat: customization for daemon deployment. Closes #223 (#290) feat: add pvc support for reduce vertex PBQ (#292) fix(doc): hyperlink for security doc (#288) feat: support adding init containers to vertices. Closes #284 (#285) fix: retry when getting EOF error at E2E test (#281) feat: Watermark millisecond. Fixes #201 (#278) feat: add pipeline node counts and age to printcolumn. Closes #267 (#282) feat: introduce reduce UDF. Closes #246 (#262) feat: add pandoc to required tools development doc. Closes

(#277) feat: add isbsvc type and age to printcolumn. Closes #268 (#275) fix: watermark consumer fix (#273) refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263) fix: set default property values for minimal CRD installation (#264) fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271) fix: main branch make build failure: math.MaxInt64 for int type (#265) fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261) fix: retry only the failed offsets (#255) fix: re-enable build constraint on processor manager test. Fixes #256 (#257) fix: container resource for jetstream isbsvc. Fixes #253 (#254) fix: update vertex watermark fetching logic. Fixes: #134 (#245) fix: watermark watcher leak (#242) fix(docs): fix a typo (#241) feat: Support running UX server with namespace scope. Fixes #248 (#249) fix(manifests): Include ServiceAccount in namespace scoped install (#240) fix: Watermark close fix and removed the nil check (#238) fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236) fix: update default watermark to -1. Fixes #133 (#218) feat: support disabling TLS and changing port for UX server (#228) feat: reducer for stream aggregation without fault tolerance (#208) feat: in-memory watermark store for better testing (#216) Add USERS.md (#221) fix(watermark): generator should not publish wm for every message (#217) David Seapy Derek Wang Ed Lee Juanlu Yu Keran Yang Shay Dratler SianLoong Vedant Gupta Vigith Maurice Yashash H L ashwinidulams dependabot[bot] xdevxy Update manifests to v0.6.5 fix: adding lock while discovering partitions, Closes #412 (#413) fix(test): e2e-api-pod can not start on M1 mac (#410) fix: getWatermark to return-1 if any processor returns -1 (#402) fix: e2e testing for PBQ WAL with reduce pipeline (#393) feat: add Grafana instruction and a dashboard template. Closes #287 (#381) fix: unit tests for replay. Closes #373 (#377) Derek Wang Keran Yang Yashash H L xdevxy Update manifests to v0.6.4 chore(docs): update docs (#380) fix: best effort processing during SIGTERM. Closes #371 (#372) feat(wal): First pass to implement WAL and hook to PBQ store. (#344) Derek Wang Vigith Maurice xdevxy Update manifests to v0.6.3 feat: watermark otwatcher enhancement (#364) refactor(docs): group docs in categories (#362) chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356) fix(controller): vertex nil check for edge listing. Fixes #352 (#353) fix: data race in pbq manager. Closes #348 (#349) Chore: Windower interface. closes #234 (#340) feat: add minikube, kind and podman support (#206) refactor: Close watermark fetcher and publisher correctly (#336) passing window information inside the context (#341) feat: timestamp in UI to display milliseconds. closes #280 (#337) Simple reduce pipeline. Fixes #289 (#317) feat: add blackhole sink. Closes #329 (#330) fix: move watermark based on the head of the read batch (#332) feat: configurable jetstream storage (#328) feat: support adding sidecars in vertex pods. Closes #323 (#325) David Seapy Derek Wang Juanlu Yu Shay Dratler Vedant Gupta Vigith Maurice Yashash H L ashwinidulams dependabot[bot] Update manifests to v0.6.2 feat: populate watermark settings to vertex spec. Closes #320 (#321) doc: add few use cases (#318) Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315) fix: jetstream build watermark progressors bug (#316) feat: update watermark offset bucket implementation (#307) feat: shuffling support (#306) feat: customize init-container resources. Closes #303 (#304) feat: watermark - remove non-share OT bucket option (#302) feat: customization for batch jobs. Closes #259 (#300) refactor: abstract pod

(#296) feat: customization for daemon deployment. Closes #223 (#290) feat: add pvc support for reduce vertex PBQ (#292) fix(doc): hyperlink for security doc (#288) feat: support adding init containers to vertices. Closes #284 (#285) fix: retry when getting EOF error at E2E test (#281) feat: Watermark millisecond. Fixes #201 (#278) feat: add pipeline node counts and age to printcolumn. Closes #267 (#282) David Seapy Derek Wang Juanlu Yu Keran Yang Vigith Maurice Update manifests to v0.6.1 fix(manifests): Include ServiceAccount in namespace scoped install (#240) fix(docs): fix a typo (#241) feat: introduce reduce UDF. Closes #246 (#262) feat: add pandoc to required tools development doc. Closes #276 (#277) feat: add isbsvc type and age to printcolumn. Closes #268 (#275) fix: watermark consumer fix (#273) refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263) fix: set default property values for minimal CRD installation (#264) fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271) fix: main branch make build failure: math.MaxInt64 for int type (#265) fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261) fix: retry only the failed offsets (#255) fix: re-enable build constraint on processor manager test. Fixes #256 (#257) fix: container resource for jetstream isbsvc. Fixes #253 (#254) fix: update vertex watermark fetching logic. Fixes: #134 (#245) fix: watermark watcher leak (#242) feat: Support running UX server with namespace scope. Fixes #248 (#249) fix: Watermark close fix and removed the nil check (#238) fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236) fix: update default watermark to -1. Fixes #133 (#218) feat: support disabling TLS and changing port for UX server (#228) feat: reducer for stream aggregation without fault tolerance (#208) feat: in-memory watermark store for better testing (#216) Add USERS.md (#221) fix(watermark): generator should not publish wm for every message (#217) David Seapy Derek Wang Ed Lee Juanlu Yu Keran Yang SianLoong Yashash H L ashwinidulams Update manifests to v0.6.0 fix(autoscaling): Ack pending should be included in total pending calculation (#212) fix(autoscaling): Skip autoscaling if vertex is not in running phase (#207) feat: ISBSVC add support for redis cluster mode (#195) refactor: move controllers package to pkg/reconciler (#192) fix: update udf fetchWatermark and publishWatermark initial values (#193) fix(docs): readme for UI development (#181) feat: grpc udsink (#174) fix: numaflow-go udf example & docs (#177) fix: use scale.max if it is set (#179) fix broken path (#176) feat: Shuffle implementation (#169) feat: windowing operations (#157) feat: watermark for sources (#159) fix: daemon service client memory leak (#161) pbq implementation (#155) feat: check if udf is running in liveness probe (#156) feat: Add udf grpc support Fixes #145 (#146) refactor: some refactor on watermark (#149) fix: Fixed JS bug (#144) bug: watermark needs nil check fix: pipeline UI broken when vertex scaling down to 0 (#132) feat: JetStream auto-reconnection (#127) feat: Add watermark for sink vertex (#124) feat: autoscaling with back pressure factor (#123) feat: add watermark to UI (#122) feat: add processing rate to UI (#121) feat: Expose watermark over HTTP (#120) fix: daemon service rest api not working (#119) chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117) feat: Numaflow autoscaling (#115) feat: watermark for headoffset (#116) feat: support namespace scope installation (#112) feat: Expose ReadTimeoutSeconds on Vertex (#110) fix: imagepullpocily for local

(#113) feat: removed udfWorkers from limits and added some docs (#103) feat: Add icon and other minor changes (#94) feat: end to end tickgen watermark validation (#98) fix: Broken hyperlink (#96) add no-op KV Store (#91) feat: no operation watermark progressor (#90) feat: kafka source pending messages (#80) feat: Interface for Watermark (#82) feat: expose pending messages and processing rate (#79) feat: Added the right way to decipher from and to vertex (#78) feat: define buffer limits on edges (#70) feat: Merge UX server code (#67) feat: auto-scaling (part 1) (#59) Added name to service spec (#58) feat: introduce source buffer and sink buffer (#53) feat: async publishing for kafka sink (#51) fix spelling errors (#48) feat: controller to create buckets (#47) turn on watermark only if ENV value is true (#46) minimal end to end line-graph watermark integration (#43) Fixed spelling error (#44) Exponential buckets (#42) fix: different behavior for time.After in go 1.18 (#39) Chrome Derek Wang Juanlu Yu Krithika3 Qianbo Huai Saravanan Balasubramanian Sidhant Kohli Vigith Maurice Yashash H L dependabot[bot] Update manifests to v0.5.6 feat: grpc udsink (#174) fix: numaflow-go udf example & docs (#177) fix: use scale.max if it is set (#179) fix broken path (#176) feat: Shuffle implementation (#169) feat: windowing operations (#157) feat: watermark for sources (#159) pbq implementation (#155) feat: check if udf is running in liveness probe (#156) feat: Add udf grpc support Fixes #145 (#146) Chrome Derek Wang Juanlu Yu Vigith Maurice Yashash H L Update manifests to v0.5.5 fix: daemon service client memory leak (#161) refactor: some refactor on watermark (#149) fix: Fixed JS bug (#144) bug: watermark needs nil check fix: pipeline UI broken when vertex scaling down to 0 (#132) Derek Wang Krithika3 Vigith Maurice Update manifests to v0.5.4 feat: JetStream auto-reconnection (#127) feat: Add watermark for sink vertex (#124) feat: autoscaling with back pressure factor (#123) feat: add watermark to UI (#122) feat: add processing rate to UI (#121) feat: Expose watermark over HTTP (#120) fix: daemon service rest api not working (#119) chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117) feat: Numaflow autoscaling (#115) feat: watermark for headoffset (#116) feat: support namespace scope installation (#112) feat: Expose ReadTimeoutSeconds on Vertex (#110) fix: imagepullpocily for local testing (#113) Derek Wang Krithika3 Saravanan Balasubramanian Sidhant Kohli Vigith Maurice dependabot[bot] Update manifests to v0.5.3 feat: removed udfWorkers from limits and added some docs (#103) feat: Add icon and other minor changes (#94) feat: end to end tickgen watermark validation (#98) fix: Broken hyperlink (#96) add no-op KV Store (#91) feat: no operation watermark progressor (#90) feat: kafka source pending messages (#80) feat: Interface for Watermark (#82) feat: expose pending messages and processing rate (#79) feat: Added the right way to decipher from and to vertex (#78) feat: define buffer limits on edges (#70) feat: Merge UX server code (#67) feat: auto-scaling (part 1) (#59) Added name to service spec (#58) Derek Wang Krithika3 Sidhant Kohli Vigith Maurice Update manifests to v0.5.2 feat: introduce source buffer and sink buffer (#53) feat: async publishing for kafka sink (#51) fix spelling errors (#48) feat: controller to create buckets (#47) turn on watermark only if ENV value is true (#46) minimal end to end line-graph watermark integration (#43) Derek Wang Qianbo Huai Vigith Maurice Update manifests to v0.5.1 Fixed spelling error (#44) Exponential buckets (#42) fix: different behavior for time.After in go 1.18 (#39) Derek Wang Krithika3