jcarbonnell/preTrainingNEAR huggingface dataset:
# Merkle Proofs

Many components of NEAR Protocol rely on Merkle root and Merkle proofs. For an array of sha256 hashes, we define its
merkle root as:
```python
CRYPTOHASH_DEFAULT = [0] * 32
def combine_hash(hash1, hash2):
    return sha256(hash1 + hash2)

def merkle_root(hashes):
    if len(hashes) == 0:
        return CRYPTOHASH_DEFAULT
    elif len(hashes) == 1:
        return hashes[0]
    else:
        l = hashes.len();
        subtree_len = l.next_power_of_two() // 2;
        left_root = merkle_root(hashes[0:subtree_len])
        right_root = merkle_root(hashes[subtree_len:l])
        return combine_hash(left_root, right_root)
```

Generally, for an array of borsh-serializable object, its merkle root is defined as
```python
def arr_merkle_root(arr):
    return merkle_root(list(map(lambda x: sha256(borsh(x)), arr)))
```

A Merkle proof is defined by:
```rust
pub struct MerklePathItem {
    pub hash: MerkleHash,
    pub direction: Direction,
}

pub enum Direction {
    Left,
    Right,
}

pub type MerkleProof = Vec<MerklePathItem>;
```

The verification of a hash `h` against a proclaimed merkle root `r` with proof `p` is defined by:
```python
def compute_root(h, p):
    res = h
    for item in p:
        if item.direction is Left:
            res = combine_hash(item.hash, res)
        else:
            res = combine_hash(res, item.hash)
    return res

assert compute_root(h, p) == r
```
```js
const ammContract = "v2.ref-finance.near";
const depositBalances = Near.view(
  ammContract,
  "get_deposits",
  {
    account_id: "bob.near"
  }
);
```

<details>
<summary>Example response</summary>
<p>

```json
{
  "token.v2.ref-finance.near": "0",
  "wrap.near": "0"
}
```

</p>

</details>
FastAuth SDK Beta Now Available
NEAR FOUNDATION
October 2, 2023
Developers can now start integrating FastAuth into their dApps for a seamless, intuitive onboarding experience with the FastAuth SDK, now in Beta.

Onboarding new users is one of the biggest hurdles for Web3 dApps. Creating a Web3 wallet is still a painful process for most users who are new to crypto, because it usually involves downloading an app or browser extension, purchasing cryptocurrency on an exchange, and understanding and saving a seed phrase. For these reasons, conversion rates for new user sign-ups tend to be very low in Web3 as compared to Web2.

What is FastAuth?
FastAuth addresses this problem through a familiar, “Web2-like” onboarding flow: users can sign up with just an email, and they don’t need to save a seed phrase or password. Instead, their FastAuth account and keys are secured using “Passkeys,” a new technology that replaces passwords with biometrics such as fingerprints or FaceID linked to a device. 

Users can also recover access to their account at any time by using their email through the “MPC recovery” service. Developers can also choose to subsidize gas costs for a certain smart contract so that users can start interacting with their dApp immediately, without having to fund their own account. 

Who can apply for the FastAuth SDK Beta?
FastAuth is currently in a closed Beta. Any developer and/or founder building NEAR apps transacting on NEAR can apply for access to the SDK. The goal of this Beta is to collect feedback from founders and developers to help inform our roadmap and upcoming releases (see more on these below). 

Pagoda will request feedback from participants in the Beta, as they start building with FastAuth.

How do I learn more and get started?
You can try FastAuth for yourself by creating a new account on near.org. 

You can also apply to be part of the beta, and check out the SDK documentation. 

What’s coming next?
Pagoda is planning to release some additional key features later this year, including the ability to “white-label” the sign-up and sign-in flows, as well as multi-chain support.
:::info
Please notice that a contract can only transfer an NFT that they own, or an NFT that they were approved to transfer.
:::

```rust
const YOCTO_NEAR: u128 = 1;

#[ext_contract(ext_nft_contract)]
trait ExternalNftContract {
  fn nft_transfer(&self, receiver_id: AccountId, token_id: TokenId) -> Promise;
}

impl Contract {
  #[payable]
  pub fn nft_transfer(&mut self, receiver_id: AccountId, token_id: TokenId) -> Promise {
    let promise = ext_nft_contract::ext(self.nft_contract.clone())
      .with_attached_deposit(YOCTO_NEAR)
      .nft_transfer(receiver_id, token_id);

    return promise.then( // Create a promise to callback query_greeting_callback
      Self::ext(env::current_account_id())
      .nft_transfer_callback()
    )
  }

  #[private] // Public - but only callable by env::current_account_id()
  pub fn nft_transfer_callback(&self, #[callback_result] call_result: Result<(), PromiseError>) {
    // Check if the promise succeeded
    if call_result.is_err() {
      log!("There was an error contacting NFT contract");
    }
  }
}
```
Case Study: Flipside Crypto on Building an Analytics Platform on the BOS
CASE STUDIES
August 2, 2023
NEAR is a home to a number of amazing apps and projects. These developers, founders, and entrepreneurs are using NEAR to effortlessly create and distribute innovative decentralized apps, while helping build a more open web — free from centralized platforms. In these Case Study videos, NEAR Foundation showcases some of these projects. 

In the latest NEAR Foundation Case Study video, we hear Flipside Crypto’s Evan “Captain” Owens and GJ Flannery. Not only is Flipside a team building innovative on-chain data and analytics solutions on NEAR, but they were one of the first ecosystem projects to build on the Blockchain Operating System (BOS). 

In this case study video, GJ Flannery (Growth & Community) and Captain Owens (Strategic Partnerships) talk about the platform’s ecosystem of analytics and developers building widgets on the BOS with Flipside’s on-chain data. If you’re a founder or developer looking to hear directly from developers who’ve built on the BOS, then this Flipside Case Study video is not to be missed.
NEAR Foundation Partners with Absolute Labs to Accelerate How Brands Connect with Customers in Web3
NEAR FOUNDATION
April 20, 2023
NEAR Foundation is excited to announce a new partnership with Absolute Labs, a company revolutionizing Web3’s marketing stack. Its “Wallet Relationship Management™ platform can be used by Web2 brands and Web3 startups alike to acquire, convert and retain customers on the Open Web. 

To help in this endeavor, the company announced an $8M seed round of funding at NFT NYC in the past week. Other funding partners include backers like Samsung Next, Aglae Ventures, Sparkle Ventures (Animoca Brands), The Luxury Fund and MoonPay. 

”We are thrilled to have the NEAR Foundation as a partner in helping us to accelerate how marketing is done in the next Internet,” says Samir Addamine, Absolute Labs’ CEO and co-founder. “Both our organizations share a vision to accelerate the world’s transition to decentralized marketplaces and more open, transparent and valued user engagement, so it’s a great fit for us.”

What is Wallet Relationship Management
Wallet Relationship Management (WRM™) starts with the premise that wallets hold the keys to everything Web3 — from proving identity, providing access to value-rich brand experiences, facilitating payments, creating “ownership” opportunities for customers, and much more. Combining on-chain data from wallets along with assets that Web2 brands can also inject from their Web2 systems, this platform supports all facets of the marketing funnel, from acquisition to conversion and retention of loyalty.

Users of the platform can gain deep insights into their customers and their on-chain preferences and behaviors, identify actionable segments to track and engage, and craft cross-chain and cross-channel (Web2 and Web3) campaigns to develop and nurture relationships

Absolute Labs customers can also use WRM to reactivate dormant wallet customers by using the WRM dashboard to distribute meaningful incentives such as rewards tokens, NFTs, and more. 

To date, Absolute Labs has 40+ companies using their services. With WRM, these customers get an enterprise-grade suite of connectors and APIs to seamlessly integrate with their existing marketing tech stack. 

Absolute Labs has an incredibly strong product and is a fantastic addition to the NEAR ecosystem. Their Wallet Relationship Management platform is a clear market leader in the burgeoning Web3 martech space, and is already an indispensable component of the modern marketing stack. By bringing Absolute Labs into the NEAR ecosystem, it creates a seamless Web3 on-ramp for brands and will enable them to engage, retain, and reactivate customers like never before.

DISCLOSURE: Any information shared in this announcement regarding our partnership with Absolute Labs should not be considered as a binding agreement or representation, and may be subject to change without notice. The information presented in this communication is for informational purposes only and should not be construed as investment advice. Any decisions made based on this information are solely the responsibility of the individual or entity making them.
<br />
<br />

<p align="center">
<img src="website/static/img/near-img.png" width="600">
</p>

<br />
<br />

## NEAR Protocol - scalable and usable blockchain

[![Discord](https://img.shields.io/discord/490367152054992913.svg)](http://near.chat)
[![CI](https://github.com/near/wiki/actions/workflows/build.yml/badge.svg)](https://github.com/near/wiki/actions/workflows/build.yml)

* ⚖️ NEAR Protocol is a new smart-contract platform that delivers scalability and usability.
* 🛠 Through sharding, it will linearly scale with the number of validation nodes on the network.
* 🗝 Leveraging WebAssembly (via Rust and AssemblyScript), more sane contract management, ephemeral accounts and many other advancements, NEAR
finally makes using a blockchain protocol easy for both developers and consumers.

## Quick start

Check out the following links

- Deployed, live Wiki: https://wiki.near.org
- Example applications: https://near.dev
- Community chat: https://near.chat

## Contributing

NEAR uses [Docusaurus](https://docusaurus.io) for documentation.  Please refer to their documentation for details on major structural contributions to the documentation.

For simple content changes you have 2 options

- [Submit an issue](https://github.com/near/wiki/issues)
- [Submit a pull request](https://github.com/near/wiki/pulls) *(we prefer PRs of course)*

### The instant PR

This is the fastest way to submit content changes directly from the page where you notice a mistake.

1. Open any page in the docs on https://wiki.near.org
2. Click the `[ Edit ]` button at the top right hand side of _every_ content page
3. Make your edits to the document that opens in GitHub by clicking the ✎ (pencil) icon
4. Submit a PR with your changes and comments for context

### The typical PR

This is the standard fork-branch-commit workflow for submitting pull requests to open source repositories

1. Fork this repo to your own GitHub account (or just clone it directly if you are currently a member of NEAR)

2. Open your editor to the _top level repo folder_ to view the directory structure as seen below

3. Move into the `/website` folder where you will run the following commands:

   - Make sure all the dependencies for the website are installed:

     ```sh
     # Install dependencies
     yarn
     ```

   - Run the local docs development server

      ```sh
      # Start the site
      yarn start
      ```

      _Expected Output_

      ```sh
      # Website with live reload is started
      Docusaurus server started on port 3000
      ```

      The website for docs will open your browser locally to port `3000` 

4. Make changes to the docs

5. Observe those changes reflected in the local docs

6. Submit a pull request with your changes
An Open Invitation to the Terra Community
COMMUNITY
May 18, 2022
The NEAR Foundation strives to create a stable environment for developers, founders and community members to realize their vision of an Open Web world.

During this turbulent period, many projects building on the Terra network are now facing an uncertain future.  The NEAR Foundation is extending an open invitation to anyone currently building on Terra to join the NEAR community, and make this vibrant ecosystem their new home.

As part of this invitation, the Foundation has put together a sizable relief fund for projects from the Terra ecosystem to make the switch. 

Why choose NEAR? 
The NEAR blockchain has simplicity at its heart. NEAR uses common coding languages like WebAssembly (WASM) in addition to Rust to build its smart contracts. Terra also uses Rust to write smart contracts, making the switch simple. 

NEAR was built by developers for developers. As a result, there are class-leading resources for developers to begin building on NEAR Protocol. These include NEAR.University’s training and certification programs—more than 5,000 developers have been trained—and $45M+ in project grants within the NEAR ecosystem. The NEAR community also has extensive documentation on building and deploying on the network, and much more.

Below is a range of places to get started: 

Developer Tools – NEAR CLIs, SDKs, APIs and more.
Code Examples – Explore how Rust and AssemblyScript are being deployed.  
Developer Bounties – Bug fixes, developer grants and other incentives. 
Grants Program – More than 800 projects funded and over $45 million awarded.
Office Hours – Talk to expert developers about your project. 
Education – Learn, watch and build.  
Ecosystem – Explore the more than 500 projects already being built in the NEAR ecosystem. 
If you’re interested in finding out more about how the NEAR Foundation team can help you make the switch, please email [email protected] with the subject line, “Terra Migration”, and one of the teams will respond within 24 hours.
---
id: community-channels
title: Node Operator Community Channels
sidebar_label: Node Operator Community Channels
sidebar_position: 1
description: Node Operator Community Channels
---

# Node Operator Community Channels {#near-node-community-channels}

NEAR Protocol will communicate with node operators using these channels. If you are running a NEAR node, please join these channels, so you will be notified.

1. Releases are announced on [Discord](https://discord.gg/ZMPr3VB), [Telegram](https://t.me/near_validators), [Twitter](https://twitter.com/NEARChainStatus), and by [email](http://eepurl.com/heLZhb). Please join the these channels, follow up along on twitter, and sign up for release email alerts here: http://eepurl.com/heLZhb.

2. Technical support and troubleshooting by the NEAR team are available in the Validator Channels on [Discord](https://discord.gg/ZMPr3VB) and on [Telegram](https://t.me/near_validators), and by the Open Shards Alliance on [Discord](https://discord.gg/t9Kgbvf).

3. Discussions on upcoming changes and early ideation are on [gov.near.org](https://gov.near.org/c/staking-delegation/5).


Releases of `nearcore` are on Github repository of [`nearcore`](https://github.com/near/nearcore/issues).

Nearcore future release dates are available on [NEAR Community Google Calendar](https://calendar.google.com/calendar/u/0/embed?src=nearprotocol.com_ltk89omsjnc2ckgbtk6h9157i0@group.calendar.google.com).





>Got a question?
<a href="https://stackoverflow.com/questions/tagged/nearprotocol">
  <h8>Ask it on StackOverflow!</h8></a>
NEAR Foundation Launches Smart Contract Audit Program
DEVELOPERS
June 21, 2022
NEAR Foundation is excited to announce that its Smart Contract Audit Program is live. This is an important step in expanding smart contract audits and enhancing ecosystem security. 

Many new projects request smart contract audits, necessitating an expansion of the number of auditors the Foundation has on retainer. This effort will make the process of choosing the most experienced and skilled auditors seamless. 

Let’s briefly explore what this means for auditors, projects, and the wider NEAR ecosystem. We’ll also explain how projects and auditors can access the program. 

Smart Contract Audits Program features
NEAR Foundation’s Smart Contract Audit Program comes with two main features—a new auditor ingest form and an audit request form. 

The New Auditor Ingest Form is the first step for interested smart contract auditors to illustrate their capabilities. It will be followed by a review process, when NEAR Foundation onboards new auditors as it expands the program. This will allow the Foundation to gain a better understanding of auditors, and have a detailed selection of auditors to choose from when deciding who to onboard.

By filling out the Smart Contract Audit Request form, projects building on NEAR can enter a queue for an audit. To file a request, your project must have a plan for funding the smart contract audit. The audit will be carried about by approved smart contract auditing firms, which NEAR Foundation has on retainer.

“The features are an important step in growing an ecosystem of trusted security partners, and ensuring availability of quality audits for NEAR-related audits,” says Isha Tyagi, NEAR Foundation’s Smart Contract Audits Program Manager. “This will also allow for the expansion and uplift of security companies in the NEAR ecosystem.” 

How to access Audit Program features
Potential auditors can find the Smart Contract Auditor ingest form here. Simply fill out your name, email, audit firm name, and other information, including past experience, capacity, and more. 

The smart contract audit request form can be found here. Fill out your name, email, and other relevant information. Be certain that your project is ready for an audit and, again, make sure you have a plan for funding your smart contract audit.
:::info
Remember about fungible token precision. You may need this value to show a response of balance requests in an understandable-to-user way in your app. How to get precision value (decimals) you may find [above](#get-token-metadata).
:::

```js
import { Wallet } from './near-wallet';

const TOKEN_CONTRACT_ADDRESS = "token.v2.ref-finance.near";
const wallet = new Wallet({ createAccessKeyFor: TOKEN_CONTRACT_ADDRESS });
 
await wallet.viewMethod({
  method: 'ft_balance_of',
  args: {
    account_id: 'bob.near'
  },
  contractId: TOKEN_CONTRACT_ADDRESS
});
```

<details>
<summary>Example response</summary>
<p>

```json
"3479615037675962643842"
```

</p>

</details>

_The `Wallet` object comes from our [quickstart template](https://github.com/near-examples/hello-near-examples/blob/main/frontend/near-wallet.js)_
---
sidebar_position: 1
---

# NEAR Nodes

NEAR Protocol runs on a collection of publicly maintained computers (or "nodes"). All nodes are running the same `nearcore` codebase with the latest releases available on [GitHub](https://github.com/near/nearcore/releases/).

It is important to keep in mind all nodes run the same codebase, with different configurations. As such, we have split up the documentation for running different types of node into sections specific to the node's type.

| Node Type | Description |
|-----------|-------------|
| [Validator Node](/validator)   | Validator nodes participate in the consensus and produce blocks and/or chunks.          |
| [RPC Node](/rpc)               | RPC nodes are service providers that provide public RPC endpoints for developers to use.           |
| [Archival Node](/archival)     | Archival nodes store full blockchain data, and build an archive of historical states.           |
---
sidebar_position: 1
sidebar_label: Introduction
pagination_label: "Promises: Introduction"
title: "Introduction"
---

# Promises

Transactions can be sent asynchronously from a contract through a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise). Like Promises in many programming languages, these will cause code to be executed in the future. In the case of NEAR, this "in the future" means a transaction to be executed _in the next block_ (or thereabouts), rather than in the same block as the original function call.

:::info Why wait?
Why not do these things synchronously, in the same block when the function is called? Why does NEAR require a `Promise` for sending tokens, or creating an account, or deploying a contract?

They need to be scheduled in separate blocks since sender and receiver accounts can be on different shards, and cross-shard communication happens across blocks by passing receipts (you can think of receipts in NEAR as "internal transactions"). You can see these receipts being passed from block to block [in NEAR Explorer](https://nearblocks.io/txns/36n3tBNiF497Tm9mijEpsCUvejL8mBYF1CEWthCnY8FV).
:::
---
id: near-lake
sidebar_label: Lake Overview
title: NEAR Lake Indexer
---

NEAR Lake is an indexer built on top of [NEAR Indexer Framework](/concepts/advanced/near-indexer-framework) to watch the network and store all the [events](../../../4.tools/events.md) as JSON files on AWS S3.

:::info GitHub repo

You can find the Lake Indexer source code in [this GitHub repository](https://github.com/near/near-lake-indexer/).

:::

### How it works

:::tip

[Pagoda Inc.](https://pagoda.co) runs NEAR Lake nodes to store the data in JSON format on AWS S3.
There is no need to run your own NEAR Lake unless you have specific reasons to do that.

:::

There are AWS S3 buckets created:

- `near-lake-data-testnet` (`eu-central-1` region)
- `near-lake-data-mainnet` (`eu-central-1` region)

All the buckets are set up the way the requester pays for the access. Anyone can read from these buckets by connecting to them with their own AWS credentials to be charged by Amazon.

### Data structure

The data structure used by Lake Indexer is the following:

```
    <block_height>/
      block.json
      shard_0.json
      shard_1.json
      ...
      shard_N.json
```

`<block_height>` is a 12-character-long [`u64`](https://doc.rust-lang.org/std/primitive.u64.html) string with leading zeros (e.g "000042839521"). See [this issue for reasoning](https://github.com/near/near-lake/issues/23).

`block_json` contains JSON-serialized `BlockView` struct. **NB!** this struct might change in the future, we will announce it

`shard_N.json` where N is [`u64`](https://doc.rust-lang.org/std/primitive.u64.html) starting from `0`. Represents the index number of the shard. In order to find out the expected number of shards in the block you can look in `block.json` at `.header.chunks_included`


### How to use it

We have created the [NEAR Lake Framework](/concepts/advanced/near-lake-framework) to have a simple straightforward way to create an indexer on top of the data stored by NEAR Lake itself.

:::info NEAR Lake Framework

You can check the NEAR Lake Framework release announcement on the [NEAR Governance Forum](https://gov.near.org/t/announcement-near-lake-framework-brand-new-word-in-indexer-building-approach/17668).

:::

We have prepared this video tutorial with a simple example to give you an overview and some practical ideas.

<iframe
 width="560"
 height="315"
 src="https://www.youtube.com/embed/GsF7I93K-EQ"
 title="NEAR Lake Indexer"
 frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
 allowfullscreen>
</iframe>