cwielenb/CNCF-Dataset huggingface dataset:
Project: Apache CarbonData (App Definition and Development / Database)
File: README.md

using makecarbonreader to read data ```bash cd python/ PYTHONPATH=/yourpath/carbondata/python/ pip install . --user ``` This creates both a `train` and `test` pycarbon datasets in `/tmp/mnist`: ```bash cd pycarbon/tests/mnist/datasetwithunischema python generatepycarbonmnist.py ``` This will invoke a 10-epoch training run using MNIST data in pycarbon form, stored by default in `/tmp/mnist`, and show accuracy against the test set: ```bash python pytorchexamplecarbon.py ``` ``` usage: pytorchexamplecarbon.py [-h] [--dataset-url S] [--batch-size N] [--test-batch-size N] [--epochs N] [--all-epochs] [--lr LR] [--momentum M] [--no-cuda] [--seed S] [--log-interval N] Pycarbon Pytorch MNIST Example optional arguments: -h, --help show this help message and exit --dataset-url S hdfs:// or file:/// URL to the MNIST pycarbon dataset (default: file:///tmp/mnist) --batch-size N input batch size for training (default: 64) --test-batch-size N input batch size for testing (default: 1000) --epochs N number of epochs to train (default: 10) --all-epochs train all epochs before testing accuracy/loss --lr LR learning rate (default: 0.01) --momentum M SGD momentum (default: 0.5) --no-cuda disables CUDA training --seed S random seed (default: 1) --log-interval N how many batches to wait before logging training status ``` This will invoke a training run using MNIST data in pycarbon form, for 100 epochs, using a batch size of 100, and log every 10 intervals. ```bash python tfexamplecarbonunifiedapi.py ``` ``` python tfexamplecarbonunifiedapi.py -h usage: tfexamplecarbonunifiedapi.py [-h] [--dataset-url S] [--training-iterations N] [--batch-size N] [--evaluation-interval N] Pycarbon Tensorflow MNIST Example optional arguments: -h, --help show this help message and exit --dataset-url S hdfs:// or file:/// URL to the MNIST pycarbon dataset(default: file:///tmp/mnist) --training-iterations N number of training iterations to train (default: 100) --batch-size N input batch size for training (default: 100) --evaluation-interval N how many batches to wait before evaluating the model accuracy (default: 10) ``` This will invoke a training run using MNIST data in pycarbon form, for 1 epoch, using a batch size of 100. ```bash python mxnetexamplecarbon.py ``` ``` python mxnetexamplecarbon.py -h usage: mxnetexamplecarbon.py [-h] [--dataset-url S] [--num-epoch N] [--batch-size N] Pycarbon Mxnet MNIST Example optional arguments: -h, --help show this help message and exit --dataset-url S hdfs:// or file:/// URL to the MNIST pycarbon dataset(default: file:///tmp/mnist) --num-epoch N the number of epoch for training (default: 1) --batch-size N input batch size for training (default: 100) ```
Project: YDB (App Definition and Development / Database)
File: run_custom.md

To run the example against any available YDB database, you need to know the and the . If authentication is enabled in the database, you also need to select the and get secrets (a token or username/password pair). Run the command as follows: ```bash <authmodevar>="<authmodevalue>" \ python3 ydb-python-sdk/examples/basicexamplev1/ -e <endpoint> -d <database> ``` where `<endpoint>`: The . `<database>`: The . `<authmodevar>`: The that determines the authentication mode. `<authmodevalue>` is the authentication parameter value for the selected mode. For example: ```bash YDBACCESSTOKEN_CREDENTIALS="t1.9euelZqOnJuJlc..." \ python3 ydb-python-sdk/examples/basicexamplev1/ -e grpcs://ydb.example.com:2135 -d /path/db ) ```
Project: Flink (App Definition and Development / Streaming & Messaging)
File: README.md

This module collects tools that help to generate test data for the state migration tests. The following dependency needs to be added to the module's Maven config in case a migration test is meant to be added to that module: ```xml <dependency> <groupId>org.apache.flink</groupId> <artifactId>fink-migration-test-utils</artifactId> <version>${project.version}</version> <scope>test</scope> </dependency> ``` and the following profile ```xml <profile> <id>generate-migration-test-data</id> <build> <plugins> <plugin> <artifactId>maven-antrun-plugin</artifactId> <executions> <execution> <id>generate-migration-test-data</id> <phase>package</phase> <goals> <goal>run</goal> </goals> <configuration> <target> <condition property="optional.classes" value="--classes '${generate.classes}'" else=""> <isset property="generate.classes"/> </condition> <condition property="optional.prefixes" value="--prefixes '${generate.prefixes}'" else=""> <isset property="generate.prefixes"/> </condition> <java classname="org.apache.flink.test.migration.MigrationTestsSnapshotGenerator" fork="true" failonerror="true" dir="${project.basedir}"> <classpath refid="maven.test.classpath"/> <arg value="--dir"/> <arg line="${project.basedir}"/> <arg value="--version"/> <arg value="${generate.version}"/> <arg line="${optional.classes}"/> <arg line="${optional.prefixes}"/> </java> </target> </configuration> </execution> </executions> </plugin> </plugins> </build> </profile> ``` To show the log during generating, add ``` logger.migration.name = org.apache.flink.test.migration logger.migration.level = INFO ``` to the `log4j2-test.properties` of this module. The state migration tests should satisfy The tests are named like `*(Test|ITCase).(java|scala)`. The test class name is the same with the file name. The test implements `org.apache.flink.test.util.MigrationTest` and the snapshots generator methods are labeled with `@SnapshotsGenerator` or `@ParameterizedSnapshotsGenerator`. To generate the snapshots for all the tests, execute from within the target version's release branch: ```shell mvn clean package -Pgenerate-migration-test-data -Dgenerate.version=1.17 -nsu -Dfast -DskipTests ``` The version (`1.17` in the command above) should be replaced with the target one. By default, it will search for the migration tests under `src/test/java` and `src/test/scala`. It is also supported to change the default search paths or only generate for specific classes: ```shell mvn clean package -Pgenerate-migration-test-data -Dgenerate.prefixes=xx,yy,zz -Dgenerate.version=1.17 -nsu -Dfast -DskipTests mvn clean package -Pgenerate-migration-test-data -Dgenerate.classes=class1,class2 -Dgenerate.version=1.17 -nsu -Dfast -DskipTests ```
Project: MySQL (App Definition and Development / Database)
File: README.md

Here we provide short but complete and working samples of some of the most important parts of the Jones API. illustrates using [session.find()] (../../database-jones/API-documentation/Session) to retreive a single record from a table. illustrates using session.persist() to store a record. illustrates using session.createQuery() to build and execute a that returns multiple records. illustrates using a to define a relationship between tables, and then running session.find() against the projection. The equivalent query in SQL would be a join. The scan and join samples use the tables in the tweet demo, described below. This directory contains a Twitter-like demo application using Database Jones. The SQL script createtweettables.sql contains DDL statements for MySQL to create the five tables used by the demo application. It can be executed using the standard mysql command: mysql -u root < createtweettables.sql The Node.js application is a rather complete large example which can run as either as a command-line tool or as a REST web server. tweet.js supplements the simple API examples with sample code for explicit transaction handling and batching (e.g. in InsertTweetOperation at line 320). tweet.js responds to a number of verb-object commands; to see a list of them, simply type: node tweet Some demonstration scripts are provided to illustrate tweet.js: populates the database with some sample data. demonstrates querying the sample data from the command shell. demonstrates querying the sample data (and posting a new tweet) over the HTTP interface. uses the HTTP interface to delete the sample data. Note that in order to run the HTTP demo scripts, you should first start the server on port 7800, using the command: node tweet start server 7800 tweet.js pays attention to the environment variables `JONES_ADAPTER` (which defaults to "ndb") and `JONES_DEPLOYMENT` (which defaults to "test"). Jones applications connect to a particular database using a named deployment defined in the file . The sample code uses the "test" deployment. You can customize jones_deployments.js for your environment, and you can supply a different deployment as a command-line option in tweet.js or by editing the call to in the API samples.
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: windowdefn.md

Operations supporting windowed aggregate functions in tremor such as the `select` statement can window incoming streams in the `from` clause: ```tremor select aggr::count(event) from in[onesecond, tensecond] ... ``` Here, we stream events from the `in` stream into a `one_second` window. Every second, we stream the aggregate result from the one second window into the `ten_second` window. So, even if we have 1 million events per second, the `onesecond` and `tensecond` windows will convert the event firehose into a `trickle`. Fun fact: this pun is where the query language got its name from.
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: patchoperationclause.md

```tremor patch {"a": 1, "b": 2, "c": 3 } of insert "d" => "delta" end; ``` It is a semantic error to insert a value if the field already exists: ```tremor Error in foo.tremor:2:3 1 | patch {"a": 1, "b": 2, "c": 3 } of 2 | insert "b" => "bravo" | ^^^^^^^^^^^^^^^^^^^^^ The key that is supposed to be written to already exists: b 3 | end; ``` ```tremor patch {"a": 1, "b": 2, "c": 3 } of update "b" => "bravo" end; ``` It is a semantic error to update a value if the field does not already exist: ```tremor 1 | patch {"a": 1, "b": 2, "c": 3 } of 2 | update "d" => "delta" | ^^^^^^^^^^^^^^^^^^^^^ The key that is supposed to be updated does not exists: d 3 | end; ``` If the distinction between an `insert` and an `update` is not significant the `upsert` operation will insert a new field, or update an existing field. This operation is more flexible, but does not offer compile time errors to protect against invalid usage. Where possible, use `insert` or `update` in preference to `upsert` when a new field or replacing an existing fields value would be an error given the business logic at hand. ```tremor patch {"a": 1, "b": 2, "c": 3 } of erase "d" end; ``` The field `c` is removed from our record ```tremor patch {"a": 1, "b": 2, "c": 3 } of move "c" => "d" # The value MUST be a string literal as it represents a field name end; ``` The `c` field is removed and a `d` field is added with the value from `c` Similar to `move`, the `copy` operation copies the value of one field to a new field ```tremor patch {"a": 1, "b": 2, "c": 3 } of copy "c" => "d" # The value MUST be a string literal as it represents a field name end; ``` The `c` field is preserved, and the `d` field is added with a copy of the value from `c` The `merge` operation in a `patch` expression can be applied to the patch target record or to a specified field. ```tremor patch {"a": 1, "b": 2, "c": 3 } of merge "d" => {} end; ``` The field `d` is created in this case with the empty record. ```tremor patch {"a": 1, "b": 2, "c": 3 } of merge => { "snot": "badger", "b": "bravo" } # This is a terse form of insert for `snot`, and `update for `b` end; ``` For repetitive or template operations, the `default` operation allows a patch record to be defined that is effectively merged with the target document ```tremor patch event of default => {"snot": {"badger": "goose"}} end ``` If event is an empty record, the result is the same as the `default` value expression. If event has a `snot` field, the value for snot is preserved. The default value is not used. Like `merge` operations, the `default` merge operation can be very effective for reducing the boilerplate complexity of patch operations and improving the readability of the transformations being performed by readers of the code. For example, we could limit our `default` record patch to apply to only the `snot` field as follows: ```tremor patch event of default "snot" => {"badger": "goose"} end ```
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: merge.md

```tremor merge {"a": 1, "b": 2, "c": 3 } of { "b": "bravo", "c": "charlie", "d": "delta" } end; ``` The merge expression loosely follows the semantics of . From our example: The field `a` is not patched and preserved The field `b` is patched with the value `bravo` - the original value is replaced The field `d` is not in the original, and is added. The expression is useful when one record with another. An alternative to the `Merge` expression is the `Patch` expression which is operation rather than value based.
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: statepath.md

Tremor programs can be stateful in many ways - such as through the state managed by operators and windowed operations by the runtime on behalf of the user provided program. The `state` keyword allows an arbitrary value controlled by a users program to be maintained and managed by the user program. ```tremor let my_state = state; ``` State can be written through via a `let` operation ```tremor let state = match state of case null => { "count": 1 } case _ => { "count"": state.count + 1 } end; ```
Project: YDB (App Definition and Development / Database)
File: LICENSE.md

Copyright (c) 2017, YANDEX LLC All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Project: Apache StreamPipes (App Definition and Development / Streaming & Messaging)
File: README.md

<!-- ~ Licensed to the Apache Software Foundation (ASF) under one or more ~ contributor license agreements. See the NOTICE file distributed with ~ this work for additional information regarding copyright ownership. ~ The ASF licenses this file to You under the Apache License, Version 2.0 ~ (the "License"); you may not use this file except in compliance with ~ the License. You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License. ~ --> This library was generated with version 12.2.0. Run `ng generate component component-name --project platform-services` to generate a new component. You can also use `ng generate directive|pipe|service|class|guard|interface|enum|module --project platform-services`. Note: Don't forget to add `--project platform-services` or else it will be added to the default project in your `angular.json` file. Run `ng build platform-services` to build the project. The build artifacts will be stored in the `dist/` directory. After building your library with `ng build platform-services`, go to the dist folder `cd dist/platform-services` and run `npm publish`. Run `ng test platform-services` to execute the unit tests via . To get more help on the Angular CLI use `ng help` or go check out the page. d> <artifactId>rocketmq-client-java</artifactId> <version>5.0.2</version> </dependency> </dependencies> <configuration> <initClass>org.apache.streampipes.extensions.all.jvm.AllExtensionsInit</initClass> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>extract-docs</goal> </goals> </execution> </executions> </plugin> ``` Replace the `version` with the current development version. Replace the `initClass` with the module's init class. The dependency to RocketMQ is only needed for modules containing the RocketMQ sink and can be omitted otherwise. The plugin creates a new folder `docs` in the module's `target` directory. The folder includes: An `img` folder which has a subdirectory for each extension (named by the `appId`) containing the icon. A `pe` folder which has a subdirectory for each extension (named by the `appId`) containing the `documentation.md` file, which has been rewritten to match the requirements of the Docusaurus Markdown parser. An updated `sidebar.json` file containing the sidebar, which is downloaded from the `streampipes-website` repo on branch `dev` and updated with the current set of extensions.
Project: Tarantool (App Definition and Development / Database)
File: README.md

These files help to prepare environment for building Tarantool statically. And builds it. Please install following tools and libraries that will be necessary for building and testing: git A C/C++ compiler. Ordinarily, this is gcc and g++ version 4.6 or later. On Mac OS X, this is Clang version 3.2+. cmake make Python and modules. Python interpreter is not necessary for building Tarantool itself, unless you intend to use the Run the test suite". For all platforms, this is python version 3.x. You need the following Python modules: pyyaml gevent six CentOS: ```bash yum install -y \ git perl gcc cmake make gcc-c++ libstdc++-static \ python3-msgpack python3-yaml python3-six python3-gevent ``` Ubuntu/Debian: ```bash apt-get install -y \ build-essential cmake make coreutils sed \ python3 python3-msgpack python3-yaml python3-six python3-gevent ``` MacOS: Before you start please install default Xcode Tools by Apple: ```bash sudo xcode-select --install sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer ``` Install brew using command from After that run next script: ```bash brew install cmake pip install --user --force-reinstall -r test-run/requirements.txt ``` ```bash cmake . make -j ctest -V ``` If you want to customise build, you need to set `CMAKETARANTOOLARGS` variable There are three types of `CMAKEBUILDTYPE`: Debug - default Release RelWithDebInfo And you want to build tarantool with RelWithDebInfo: ```bash cmake -DCMAKETARANTOOLARGS="-DCMAKEBUILDTYPE=RelWithDebInfo" . ```
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: api-report-alpha.md

Do not edit this file. It is a report generated by . ```ts import { BackendFeature } from '@backstage/backend-plugin-api'; import { CatalogCollatorEntityTransformer } from '@backstage/plugin-search-backend-module-catalog'; import { ExtensionPoint } from '@backstage/backend-plugin-api'; // @alpha export type CatalogCollatorExtensionPoint = { setEntityTransformer(transformer: CatalogCollatorEntityTransformer): void; }; // @alpha export const catalogCollatorExtensionPoint: ExtensionPoint<CatalogCollatorExtensionPoint>; // @alpha const _default: () => BackendFeature; export default _default; // (No @packageDocumentation comment for this package) ```
Project: Apache Ignite (App Definition and Development / Database)
File: README.md

Templates for the `dotnet new` command. `public` templates are published to NuGet for the end users Template for the entire `../examples/Apache.Ignite.Examples.sln` solution - an easy way to deliver examples to the end user. Built as part of `build.ps1` Published to NuGet during the release Use: `dotnet new ignite-examples` `internal` templates are only used locally and not published Template for Server and Thick Client examples. Install: `dotnet new --install internal/Apache.Ignite.Example` Use: `dotnet new ignite-example` Template for Thin Client examples. Install: `dotnet new --install internal/Apache.Ignite.ExampleThin` Use: `dotnet new ignite-example-thin`
Project: Numaflow (App Definition and Development / Streaming & Messaging)
File: README.md

A web-based UI for Numaflow The UI has the following features: View running pipelines in your namespace View Vertex and Edge Information of your pipeline View BackPressure and Pending Messages View Container Logs for a given vertex See . ```shell make start kubectl -n numaflow-system port-forward svc/numaflow-server 8443 ``` Import the App component the run the package as follows: CJS ```javascript import App from 'numaflow-ui-test/dist/cjs/src/components/plugin/NumaflowMonitorApp/App'; ``` ESM ```javascript import App from 'numaflow-ui-test/dist/esm/src/components/plugin/NumaflowMonitorApp/App'; ``` Pass the cluster hostname and namespace to get the UI running ```javascript <App hostUrl={hostUrl} namespace={namespace} /> ```
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: doccomment.md

Documentation level comments are used throughout the tremor standard library and used as part of our document generation process. Here is a modified snippet from the standard library to illustrate ```tremor intrinsic fn instance() as system::instance; ... ``` This is a builtin function implemented in rust and used in a script as follows: ```tremor use tremor::system; system::instance() ```
Project: YugabyteDB (App Definition and Development / Database)
File: explore-ycql.md

Run `ycqlsh` to connect to the service. You can do this as shown below. ```sh $ docker exec -it yb-tserver-n1 /home/yugabyte/bin/ycqlsh yb-tserver-n1 ``` ```output Connected to local cluster at 127.0.0.1:9042. [ycqlsh 5.0.1 | Cassandra 3.9-SNAPSHOT | CQL spec 3.4.2 | Native protocol v4] Use HELP for help. ycqlsh> ``` Run a YCQL command to verify it is working. ```cql ycqlsh> describe keyspaces; ``` ```output systemschema systemauth system ycqlsh> ```