vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This is the only way to connect a transit gateway to a Classic Access VPC: by connecting the entire classic infrastructure to the transit gateway (instead of the specific Classic Access VPCs).







 Generic Routing Encapsulation (GRE) connection considerations 

When configuring a GRE tunnel, you must specify an availability zone in which to create the tunnel. Because of this, if for some reason that zone becomes unavailable, any network connected through a GRE tunnel on that zone is unreachable. To configure a highly available GRE tunnel, you must create a GRE tunnel in multiple zones, connecting the same endpoints.

GRE connections require the use of a BGP service between GRE tunnel IP addresses. The transit gateway configures a BGP service on the tunnel connection, before connecting to the other tunnel endpoint. Then, once the BGP protocol exchanges routes between the connected endpoint and the transit gateway, the GRE tunnel becomes the data path for the routed traffic.

GRE Tunnel routes are learned directly on BGP sessions established over the tunnel. For this reason, prefix filtering is not enabled for these connections.

The number of GRE tunnels connected to a transit gateway is quota limited. The default quota is 12.

Legacy GRE tunnel considerations:



* Classic routes are not advertised through a traditional GRE tunnel.
* Legacy GRE tunnels cannot communicate through other GRE tunnels on the same transit gateway.
* Legacy GRE tunnels require a classic connection on the transit gateway before creation. As a result, all classic subnets will be advertised to all connections attached to the transit gateway, as well as any other of the connection's subnets on the classic network.



Unbound GRE tunnel Considerations:



* Classic routes are advertised through an unbound GRE tunnel.
* Unbound GRE tunnels can communicate through other unbound GRE tunnels connected to the same transit gateway in the same availability zone.
* Unbound GRE tunnels cannot communicate with other unbound GRE tunnels on the same transit gateway if they are in a different availability zone. Unbound GRE tunnels in this scenario cannot be relied on for network isolation.



If you require network isolation, consider using separate transit gateways.
In this tutorial, we have combined a number of IBM services to model a realistic IoT pattern where Internet-enabled devices on trucks are producing data that gets consumed by various applications for different purposes.

Because of the managed nature of all these services, as an application developer, you can focus on solving your business problems and leave us to do the heavy lifting of managing the infrastructure for you, thereby saving potentially months of work on non-core activities like server provisioning, software deployment, patching, security and others.

If you followed this tutorial, remember to de-provision your infrastructure to stop incurring charges. On your terminal, do the following:

cd terraform/
terraform destroy

If you want to take the next step in your developer journey, check out some of our [trial offers](https://www.ibm.com/products?types%5B0%5D=trial).
transport_protocol: int,
ether_type: string,
was_initiated: boolean,
was_termined: boolean,
bytes_from_initiator: long,
packets_from_initiator: long,
bytes_from_target: long,
packets_from_target: long,
cumulative_bytes_from_initiator: long,
cumulative_packets_from_initiator: long,
cumulative_bytes_from_target: long,
cumulative_packets_from_target: long
>>,
account string,
region string,
vpc-id string,
subnet-id string,
endpoint-type string,
instance-id string,
vnic-id string,
record-type string,
year int,
month int,
day int,
hour int,
stream-id string
) USING JSON
LOCATION cos://<region>/<bucket>/ibm_vpc_flowlogs_v1/
Show more
2. Define a view for convenient flow log queries:

CREATE VIEW FLOW_FLAT AS
WITH EXPLODED_FLOW as (
SELECT version,collector_crn,attached_endpoint_type,network_interface_id, instance_crn, capture_start_time, capture_end_time, vnic-id, record-type, year, month, day, hour, stream-id, explode(flow_logs) as flow FROM FLOW)
SELECT version, collector_crn, attached_endpoint_type, network_interface_id, instance_crn, capture_start_time, capture_end_time, vnic-id, record-type, year, month, day, hour, stream-id, flow.
FROM EXPLODED_FLOW



You can verify that flow log data is being collected by using IBM Cloud Monitoring. For more information, see [Monitoring flow logs for VPC metrics](https://cloud.ibm.com/docs/vpc?topic=vpc-fl-monitoring-metrics).



 Optimizing flow logs layout with IBM Cloud® Data Engine
[tgw-classic](https://cloud.ibm.com/docs-content/v1/content/d50364af907081c3c38f99c11f35dd9cab4b2510/vpc-journey/images/tgw/vpc-to-classic-tgw.png)

The following example depicts how Transit Gateway devices can be used to connect VPCs together with Direct link.

![tgw-directlink](https://cloud.ibm.com/docs-content/v1/content/d50364af907081c3c38f99c11f35dd9cab4b2510/vpc-journey/images/tgw/directlink-tgw.png)

For additional details on how to configure a Transit Gateway, along with some scenario, please refer to the [documentation](https://cloud.ibm.com/docs/transit-gateway?topic=transit-gateway-about)





 Next Steps 

The next step on the deployment journey is:



* [Setup VPN Connectivity](https://cloud.ibm.com/docs/vpc-journey?topic=vpc-journey-vpc-vpn)
[Firewall](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/vpc-transit/vpc-transit-firewall.svg)

Firewall

Connectivity from the enterprise to a spoke is achieved through a Network Function Virtualization, [NFV](https://cloud.ibm.com/docs/vpc?topic=vpc-about-vnf), firewall-router instance in the transit VPC. In production you can choose one from the catalog or bring your own. This demonstration will use an Ubuntu stock image with kernel iptables set up to forward all packets from the source to destination. In this tutorial, no firewall inspection is actually performed.

The Terraform configuration will configure the firewall-router instance with [allow_ip_spoofing](https://cloud.ibm.com/docs/vpc?topic=vpc-ip-spoofing-about). You must [enable IP spoofing checks](https://cloud.ibm.com/docs/vpc?topic=vpc-ip-spoofing-aboutip-spoofing-enable-check) before continuing.



1. Apply the firewall_tf layer:

./apply.sh firewall_tf
2. Run the test suite.

Your expected results are: Connectivity within a VPC, enterprise -> transit, enterprise <-> spoke same zone pass. But all transit -> spoke, all transit -> enterprise and enterprise <--> spoke cross zone fail due to asymmetric routing issues.

pytest -m "curl and lz1 and (rz1 or rz2)"

[Part two](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-vpc-transit2) of this tutorial will route all VPC <-> different VPC traffic through the router and resolve these issues.
* Direct Link can be attached as a connection to the Transit Gateway.
* Transit gateway GRE tunnels support traffic flows:



* From GRE to VPC and from VPC to GRE, and
* From GRE to Direct Link and from Direct Link to GRE.



* Plan your AS numbering before you order the GRE tunnels to Transit Gateway.
* To speed up recovery in failure situations, you can tune BGP keepalive and hold-time values at your end of the GRE tunnel. These values are the main mechanisms that the Transit Gateway's BGP uses to ensure that its BGP neighbors are still alive through the GRE tunnel. [Transit gateway router](https://cloud.ibm.com/docs/transit-gateway) provides its default keepalive and hold-time values for the BGP session inside the [GRE tunnel](https://cloud.ibm.com/docs/transit-gateway?topic=transit-gateway-GRE-connection&interface=ui), but you can provide smaller timer values on your side of the GRE tunnel.



When a BGP connection is established with the local routing device, a peer sends an open message that contains a hold-time value. If the router does not receive successive BGP messages, such as keepalive or update within the period that is specified in hold-time the BGP connection is closed. The BGP hold-time is three times the interval at which keepalive messages are sent, and hold-time is the maximum number of seconds allowed to elapse between successive keepalive messages that BGP receives from a peer. BGP on the local routing device uses the smaller of either the local hold-time value or the peer’s hold-time value as hold-time for the BGP connection between the two peers.

Bidirectional Forwarding Detection (BFD) is not supported by Transit gateway's GRE tunnels.

In vSRX, the default hold-time is 90 seconds, which means that the default frequency for keepalive messages is 30 seconds.
[Sample Direct Link on-premises interconnect use case](https://cloud.ibm.com/docs-content/v1/content/f02b9c9adcbb7328d95a45e105cec371d50f15eb/vpc//images/direct-link-dedicated.png)

Example Direct Link on-premises interconnect use case
* IBM Cloud Direct Link Connect provides connectivity between your on-premises and IBM Cloud VPC networks through a supported service provider. A service provider connection is useful if your data center is in a physical location that can't reach a dedicated colocation facility, or if your data needs don't warrant a 5 Gbps+ connection. Connect service providers are often used to facilitate multicloud connectivity (public clouds from multiple vendors) through their network. Connect service providers offer layer 2 connectivity, layer 3 connectivity, or both. Work with your service provider to understand their offerings and requirements.



* VPN for VPC can securely connect your virtual private cloud to another private network. You can use VPN to set up an IPsec site-to-site tunnel between your VPC and your on-premises private network or another VPC. See [Connecting to your on-premises network using a VPN gateway](https://cloud.ibm.com/docs/vpc?topic=vpc-vpn-onprem-example) for details.







 Interconnecting VPCs 

IBM Cloud Transit Gateway provisions and defines connections between resources on the IBM Cloud network, providing private interconnectivity between IBM Cloud data centers worldwide. IBM Cloud Transit Gateway provides a central hub for connectivity, making it easier to provision and manage your networks. With IBM Cloud Transit Gateway, you can create a single transit gateway or multiple transit gateways to connect IBM Cloud VPCs. You can also connect your IBM Cloud classic infrastructure to a transit gateway to provide seamless communication with classic infrastructure resources. Any new resource that you connect to a transit gateway is automatically made available to every other resource connected to it. All data remains within the private IBM Cloud backbone and is optimized for performance.

Zoom

!
The following response results show once you initiate the request:



Table 3. Initating request response

 Response body Details 

 name <br>Always included* <br>Name The user-defined name for this transit gateway connection. <br>Possible values: 1 ≤ length ≤ 63, Value must match regular expression `^([a-zA-Z] 
 network_type <br>Always included* <br>string Defines what type of network is connected via this connection. The list of enumerated values for this property may expand in the future. Code and processes using this field must tolerate unexpected values. <br>Possible values: [gre_tunnel] <br>Example:gre-tunnel 
 id <br>Always included* <br>string The unique identifier for this Transit Gateway Connection <br>Example:1a15dca5-7e33-45e1-b7c5-bc690e569531 
 created_at <br>Always included* <br>date-time The date and time that this connection was created 
 base_connection_id <br>string network_type gre_tunnel connections use base_connection_id to specify the ID of a network_type classic connection the tunnel is configured over. The specified connection must reside in the same transit gateway and be in an active state. The classic connection cannot be deleted until any gre_tunnel connections using it are deleted. This field only applies to and is required for network type gre_tunnel connections. <br>Example:975f58c1-afe7-469a-9727-7f3d720f2d32 
 local_bgp_asn <br>integer Local network BGP ASN. This field only applies to network type gre_tunnel and unbound_gre_tunnel connections. <br>Example:64490 
 local_gateway_ip <br>string Local gateway IP address. This field only applies to network type gre_tunnel and unbound_gre_tunnel connections. <br>Example:192.168.100.1 
 local_tunnel_ip <br>string Local tunnel IP address.
The Transit Gateway delivers to the transit VPC using the matching address prefix 192.168.0.0/24 in the upper zone. The IBM Cloud VPC uses the industry standard state based routing for secure TCP connection tracking. This requires that the TCP connection pass through the same firewall-router in both directions. The VPC does not support tcp "Asymmetric Routing".

It is interesting to note that an attempt to ping using the ICMP protocol would not suffer from this limitation. ICMP does not require a stateful connection. Connectivity from 192.168.0.4 <--ICMP--> 10.2.1.4 via ICMP is possible. You can run the ping marked tests to verify via copy paste of the failed output. The l- is for left and r- is for right:

Expect success:

pytest -m ping -k 'l-enterprise-z1 and r-spoke0-z2'

Expect failure:

pytest -m curl -k 'l-enterprise-z1 and r-spoke0-z2'

If the goal is to create an architecture that is resilient across IBM Cloud zonal failures then cross zone traffic should generally be avoided. Routing on the enterprise could insure that all traffic destined to the cloud be organized and routed to avoid the cross zone traffic in the cloud. The enterprise concept of zones will need to be understood. In this tutorial the phantom VPC address prefixes will identify the zone associated with an enterprise device.





 Spoke Egress routing 

To resolve this problem transit <-> spoke(s) traffic will be routed to stay in the same zone. The spoke -> transit traffic can be routed using an egress routing table in the spokes.

Zoom

![Routing traffic from spoke to transit with an egress routing table](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/vpc-transit/vpc-transit-spoke-egress.svg)
Capacity management handles the overall available capacity on the transit gateway and is subject to our weekly capacity management review. When the device reaches roughly a 50% load, we augment the connectivity to the device.





 What scalability options do I have for my transit gateway? Does it manage itself? How do I know if it's reaching maximum capacity? 

The IBM Cloud infrastructure manages all transit gateways. There are no scalability options available.





 How do you prevent Distributed Denial of Service (DDoS) attacks? What restrictions do you have in place? 

Neither third-parties nor the internet can see your transit gateway traffic. As no critical information, such as IP router addresses, is open to anyone but you, DDoS attacks cannot bring down the network. In addition, a typical Multi-protocol Label Switching service (MPLS) uses packet filtering and applies access control lists (ACLs) to limit access. Only the ports with routing protocols from a specific area of the network can access the information.





 How does my transit gateway handle encryption for connectivity between VPCs? 

IBM Cloud Transit Gateway does not perform encryption; it only provides connectivity. Encryption between VPCs is your own responsibility.

It is an RFC-2547-based platform where the core network and network address are 100% concealed.





 What are the tools for monitoring the consumption of resources associated with the service, as well as the costs and the quality of the service? 

IBM Cloud Transit Gateway is integrated into the [IBM Cloud usage dashboard](https://cloud.ibm.com/docs/billing-usage?topic=billing-usage-viewingusage), which provides a summary of estimated charges for all services and resources that are used per month in your organizations. This includes the number of connections and the amount of traffic flowing across your transit gateways. IBM Cloud Transit Gateway usage is billed and reported as part of the [IBM Cloud invoice process](https://cloud.ibm.com/docs/billing-usage?topic=billing-usage-managing-invoices).





 Are there notifications through email for events of unavailability of the service?
[Using Calico network policies to control traffic on Classic clusters](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutorialpolicy_tutorial)



* [Objectives](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutorialpolicies_objectives)
* [Audience](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutorialpolicies_audience)
* [Prerequisites](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutorialpolicies_prereqs)
* [Deploy an app and expose it by using an NLB](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutoriallesson1)
* [Block all incoming traffic to all node ports](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutoriallesson2)
* [Allow incoming traffic from a specific IP to the NLB](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutoriallesson3)
* [Deny incoming traffic from specific IPs to the NLB](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutoriallesson4)
* [Logging blocked traffic from specific IPs to the NLB](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutoriallesson5)
* [What's next?](https://cloud.ibm.com/docs/containers?topic=containers-policy_tutorialwhats_next)



[Controlling traffic with network policies](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesnetwork_policies)



* [Default Calico and Kubernetes network policies](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesdefault_policy)
Centralize communication through a VPC Transit Hub and Spoke architecture - Part two 

This tutorial may incur costs. Use the [Cost Estimator](https://cloud.ibm.com/estimator/review) to generate a cost estimate based on your projected usage.

A Virtual Private Cloud (VPC) provides network isolation and security in the IBM Cloud. A VPC can be a building block that encapsulates a corporate division (marketing, development, accounting, ...) or a collection of microservices owned by a DevSecOps team. VPCs can be connected to an on-premises enterprise and each other. This may create the need to route traffic through centralized firewall-gateway appliances. This tutorial will walk through the implementation of a hub and spoke architecture depicted in this high-level view:

Zoom

![vpc-transit-overview](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/vpc-transit/vpc-transit-overview.svg)

Figure 1. Architecture diagram of the tutorial

This is part two of a two part tutorial. This part will focus on routing all traffic between VPCs through a transit hub firewall-router. A scalable firewall-router using a Network Load Balancer is discussed and implemented. Private DNS is used for both for microservice identification and IBM Cloud service instance identification using a virtual private endpoint gateway.

This tutorial is stand alone and it is not required to execute the steps in [part one](https://cloud.ibm.com/docs/solution-tutorials?topic=solution-tutorials-vpc-transit1). If you are not familiar with VPC, network IP layout and planning in the IBM Cloud, Transit Gateway, IBM Cloud® Direct Link or asymmetric routing consider reading through part one.

The hub and spoke model supports a number of different scenarios:



* The hub can be the repository for shared micro services used by spokes and enterprise.
The control plane high availability is discussed in another availability pattern.

The main reason that you need two layers is for physical north-south connectivity in the data centers. Each data center must have their own private and public VLANs and IP addresses for the uplinks. These IP addresses cannot move between the data centers. Therefore, it requires your attention for routing and network address translation if you use public connectivity in this topology.





 Multisite – multitenant 

Multisite – single tenant is also a use case and network deployment pattern where tenants require a solution with conflicting IP spaces and complete route table isolation. This topology provides seamless site recovery by using dynamic routing protocols, such as BGP. The vCenter Server automation doesn't deploy multisite topology automatically. However, you can customize the default single-site topology post initial vCenter Server deployment. When you have the compute capacity in the required data center, you can deploy the required extra edge nodes manually. Then, you can create a new edge cluster for the Tier-0 and Tier-1 gateways through NSX-T. This overlay topology is highly scalable and it is possible to automate, for example by using Ansible and Terraform.

This option is suitable for customers with workloads that run on a multiple data center, single tenant, and have overlapping IP addresses inside their own workloads.

Overlapping IP addresses refers to overlay segments between customer environments that run on the vCenter Server instance.

The following diagram shows an example of a multisite – multitenant topology. It consists of a two layer Tier-0 design with three edge clusters. The routing table separation is done at Tier-1 and optionally also at the regional Tier-0 level. For this, consider two edge clusters in the two data centers or zones in a multizone region. And also, one edge cluster deployed across the multizone region with edge nodes in each participating zone or data center.

Zoom

!
This allows for a cascading HCX migration plan where groups of VMs by VLAN are migrated and the L2 networks they reside upon are stretched only until the point the VLANs are released.



This means that the initial group of related L2 stretched networks can be unstretched only when the cloud side network design is finalized and deployed. Unstretching implies swinging the particular VXLAN traffic to now route through the cloud instance NSX infrastructure.





 Baseline network configuration 

Create a secured perimeter network within the cloud side vSphere instance. This usually consists of an NSX DLR or Edge appliance. If you use HCX Proximity Routing, you do not need to create any firewall rules or uplink topology as it can be completed later or concurrently without effecting the stretched L2 traffic.





 Network extension 

Extending the network merely means to take the existing VLAN or VXLAN from the source vSphere environment, represented by a virtual distributed switch port group (vDS), and extending it to an NSX VXLAN on the cloud side of HCX.





 Pre-flight tests 

Pre-flight tests involve performing an HCX migration with both the vMotion and bulk migration function to establish a baseline transfer rate.





 Migration of nonproduction apps 

Migration of VMs begins with the planned stages of less critical VMs. Development and Test teams use internet connectivity for migration and stretched L2 traffic.





 Cloud network design and implementation begins 

While migrations continue, cloud side network designs are finalized and implemented within the cloud side vSphere instance.





 More network connectivity considerations 

While migrations continue, private WAN network connectivity is ordered as it typically takes a few weeks to months to be established with the cloud provider. After private network connectivity is completed, HCX can be configured to use both, the dedicated private network link and the internet for migration and stretched L2 traffic.





 Physical servers 

When the goal is data center migration into the cloud, any physical servers that interact with the VMs being migrated can be assessed for migration into the IBM Cloud as either a VM (P2V), bare metal or remain at the source.