AI-Mock-Interviewer/PDF_Extracted_Data huggingface dataset:
Question: how do we measuring the impact of garbage collection?
Answer: GC has happened due to use too much of memory on a driver or some executors or it might be where garbage collection becomes extremely costly and slow as large numbers of objects are created in the JVM. You can do by this validation ' -verbose:gc -XX:+ PrintGCDetails -XX:+PrintGCTimeStamps' to Sparks JVM options using the `spark.executor.extraJavaOptions` configuration parameter.
Question: apache spark vs apache storm?
Question: how to overwrite the output directory in spark?
Answer: refer below command using Dataframes, df.write.mode(SaveMode.Overwrite).parquet(path)
Question: how to read multiple text files into a single rdd?
Answer: You can specify whole directories,  use wildcards and even CSV of directories and wildcards like below. Eg.: val rdd = sc.textFile("file:///D:/Dinesh.txt, file:///D:/Dineshnew.txt")
Question: can we runs park without base of hdfs?
Question: define about generic classes in scala?
Question: how to enable tungsten sort shuffle inspark2x?
Question: how to prevent spark executors from getting lost when using yarn client mode?
Answer: The solution if you're using yarn was to set --conf spark.yarn.executor.memoryOverhead=600, alternatively if your cluster uses mesos you can try --conf spark.mesos.executor.memoryOverhe ad=600 instead.
Question: what is the relationship between the yarn containers and the spark executors?
Answer: First important thing is this fact that the number of containers will always be the same as the executors created by a Spark application e.g. via --num-executors parameter in spark -submit. Set by the yarn.scheduler.minimum -allocation -mb every container always allocates at least this amount  of  memory.  This  means  if  parameter --executor -memory  is  set  to  e.g.  only 1g  but yarn.scheduler.minimum -alloca tion-mb is e.g. 6g, the container is much bigger than needed by the Spark application. The other way round, if the parameter --executor -memory is set to somthing higher than the yarn.scheduler.minimum -allocation -mb value, e.g. 12g, the Container will alloc ate more memory dynamically,   but   only   if   the   requested   amount   of   memory   is   smaller   or   equal   to yarn.scheduler.maximum -allocation -mb value. The  value  of  yarn.nodemanager.resource.memory -mb  determines,  how  much  memory  can  b e allocated in sum by all containers of one host! So setting yarn.scheduler.minimum -allocation -mb allows you to run smaller containers e.g. for smaller executors (else it would be waste of memory). Setting   yarn.scheduler.maximum -allocation -mb   to   the    maximum   value  (e.g.   equal   to yarn.nodemanager.resource.memory -mb) allows you to define bigger executors (more memory is Follow  Me : https://www.yo utube.com/c/SauravAgarwa l allocated if needed, e.g. by --executor -memory parameter).
Question: how to allocate the memory sizes for the spark jobs in cluster?
Question: how autocompletion tab can enable in py spark?
Answer: Please import the below libraries in pyspark shell import rlcompleter, readline readline.parse_and_bind("tab: complete")
Question: can we execute two transformations on the same rdd in parallel in apache spark?
Question: which cluster type should i choose for spark?
Question: what is d streams in spark streaming?
Answer: Spark streaming uses a micro batch archite cture where the incoming data is grouped into micro batches called Discretized Streams (DStreams) which also serves as the basic programming abstraction. Follow  Me : https://www.yo utube.com/c/SauravAgarwa l The DStreams internally have Resilient Distributed Datasets (RDD) and as a result of this standard RDD transformations and actions can be done.
Question: what is stateless transformation?
Question: what is stateful transformation?
Question: what is a ws?
Answer: Follow  Me : https://www.yo utube.com/c/SauravAgarwa l Answer:AWS  stands for Amazon Web Services. AWS is a platform that provides on -demand resources for hosting web services, storage, networking, databases and other resources over the internet with a pay -as-you-go pricing.
Question: what are the components of aws?
Answer: Answer:EC2   Elastic Compute Cloud, S3  Simple Storage Service, Route53, EBS  Elastic Block Store, Cloudwatch, Key -Paris are few of the components of AWS.
Question: what are key pairs?
Answer: Answer:Key -pairs are secure login information for your instances/virtual machines. To  connect to the instances we use key -pairs that contain a public -key and private -key.
Question: what is s3?
Question: what are the pricing models for ec2 instances?
Answer: Answer:The different pricing model for EC2 instances are as below,  On-demand  Reserved  Spot  Schedu led  Dedicated
Question: what are the types of volumes for ec2 instances?
Question: what are ebs volumes?
Answer: Answer:EBS  stands for Elastic Block Stores. They are persistent volumes that you can attach to the instances. With EBS volumes, your data will be preserved even when you stop your instances, unlike your instance store volumes where the data is deleted when you stop the instances.
Question: what are the types of volumes in ebs?
Question: what are the different types of instances?
Answer: Answer: Following are the typ es of instances,  General purpose  Computer Optimized  Storage Optimized  Memory Optimized  Accelerated Computing
Question: what is an auto scaling and what are the components?
Answer: Answer: Auto scaling allows you to automatically scale -up and scale -down the number of instances depending on the CPU utilization or memory utilization. There are 2 components in Auto scaling, they are Auto -scaling groups and Launch Configuration.
Question: what are reserved instances?
Question: what is an ami?
Question: what is an eip?
Question: what is cloud watch?
Answer: Answer: Cloudwatch is a monitoring tool that you can use to monitor your various AWS resources. Like health check, network, Application, etc.
Question: what are the types in cloud watch?
Answer: Answer: There are 2 types in cloudwatch. Basic monitoring and detailed monitoring. Basic monitoring is free and detailed monitoring is chargeable.
Question: what are the cloud watch metrics that are available for ec2 instances?
Question: what are the different storage classes in s3?
Answer: Answer: Following are the types of storage classes in S3,  Standar d frequently accessed  Standard infrequently accessed  One-zone infrequently accessed.  Glacier  RRS  reduced redundancy storage
Question: what is the default storage class ins3?
Answer: Answer: The default storage class in S3 in Standard frequently accessed.
Question: what is glacier?
Answer: Answer: Glacier is the back up or archival tool that you use to back up your data in S3.
Question: how can you secure the access to your s3 bucket?
Answer: Answer: There are two ways that you can control the access to your S3 buckets,  ACL  Access Con trol List  Bucket polices
Question: how can you encrypt data in s3?
Answer: Answer: You can encrypt the data by using the below methods,  Server Side Encryption  S3 (AES 256 encryption)  Server Side Encryption  KMS (Key management Service)  Server Side  Encryption  C (Client Side)
Question: what are the parameters for s3 pricing?
Answer: Answer: The pricing model for S3 is as below,  Storage used  Number of requests you make  Storage management  Data transfer  Transfer acceleration
Question: what is the prerequisite to work with cross region replication in s3?
Answer: Answer: You need to enable versioning on both source bucket and destination to work with cross region replication. Also both the source and destination bucket should be in different region.
Question: what are roles?
Answer: Follow  Me : https://www.yo utube.com/c/SauravAgarwa l Answer: Roles are used to provide permissions to entities that you trust within your AWS account. Roles are users in another account. Roles are similar to users but with roles you do not need to create any username and password to work with the resources.
Question: what are policies and what are the types of policies?
Question: what is cloud front?
Answer: Answer: Cloudfront is an AWS web service that provided businesses and application developers an easy and efficient way to distribute their content with low latency a nd high data transfer speeds. Cloudfront is content delivery network of AWS.
Question: what are edge locations?
Question: what is the maximum individual archive that you can store in glacier?
Answer: Answer: You can store a maximu m individual archive of upto 40 TB.
Question: what is vpc?
Question: what is vpc peering connection?
Question: what are nat gateways?
Answer: Answer: NAT stands for Network Address Translation. NAT gateways enables instances in a private subnet to connect to the internet but prevent the internet from initiating a connection with those instances.
Question: how can you control the security to your vpc?
Answer: Answer: You can use security groups and NACL (Network Access Control List) to control the security to your VPC.
Question: follow me https www you tubecomcsauravagarwalq34 what are the different types of storage gateway?
Answer: Answer: Following are the ty pes of storage gateway.  File gateway  Volume gateway  Tape gateway
Question: what is a snowball?
Answer: Answer: Snowball is a data transport solution that used source appliances to transfer large amounts of data into and out of AWS. Using snowball, you can move huge amount of data from one place to another which reduces your network costs, long transfer time s and also provides better security.
Question: what are the database types in rds?
Answer: Answer: Following are the types of databases in RDS,  Aurora  Oracle  MYSQL server  Postgresql  MariaDB  SQL server
Question: what is a redshift?
Question: what issns?
Answer: Answer: SNS stands for Simple Notification Service. SNS is a web service that makes it easy to notifications from the cloud.  You can set up SNS to receive email notification or message notification.
Question: what are the types of routing police sinroute53?
Answer: Answer: Following are the types of routing policies in route53,  Simple routing  Latency routing  Failover routing  Geolocation r outing  Weighted routing  Multivalue answer
Question: what is the maximum size of messages in sqs?
Answer: Answer: The maximum size of messages in SQS is 256 KB.
Question: what are the types of queues in sqs?
Question: what is multi az rds?
Answer: Answer: Multi -AZ (Availability Zone) RDS allows you to have a replica of your production database in another availability zone. Multi -AZ (Availability Zone) database is used for disaster recovery. You will have an exact copy of your database. So when your primary database goes down, your application will automatically failover to the standby database.
Question: what are the types of backups in rds database?
Answer: Answer: There are 2 types of backups in RDS database .  Automated backups  Manual backups which are known as snapshots.
Question: what is the difference between security groups and network access control list?
Answer: Answer: Security Groups  Network access control list Can control the access at the instance level  Can control access at the subnet level Can add rules for allow only  Can add rules for both allow and deny Evaluates all rules before allowing the traffic  Rules are processed in order number when allowing traffic. Can assign unlimited number of securit y groups  Can assign upto 5 security groups. Statefull filtering  Stateless filtering
Question: what are the types of loadbalancer sinec2?
Answer: Answer: There are 3 types of load balancers,  Application load balancer  Network load balancer  Classic load balancer
Question: what is an del b?
Question: what are the two types of access that you can provide when you are creating users?
Answer: Answer: Following are the two types of access that you can create.  Programmatic access  Console access
Question: what are the benefits of auto scaling?
Answer: Follow  Me : https://www.yo utube.com/c/SauravAgarwa l Answer: Following are the benefits of auto scaling  Better fault tolerance  Better availability  Better cost management
Question: what are security groups?
Answer: Answer: Security groups acts as a firewall that contains the traffic for one or more instances. You can associate one or more security groups to your instances when you launch then. You can add rules to each security group that allow traffic to and from it s associated instances. You can modify the rules of a security group at any time, the new rules are automatically   and immediately applied to all the instances that are associated with the security group
Question: what are shared am is?
Question: what is the difference between the classic loadbalancer and application loadbalancer?
Answer: Answer: Dynamic port mapping, multiple  port multiple listeners is used in Application Load Balancer, One port one listener is achieved via Classic Load Balancer
Question: by default how many ip address does aws reserve in a subnet?
Answer: Answer: 5
Question: what is meant by subnet?
Question: how can you convert a public subnet to private subnet?
Question: is it possible to reduce a ebs volume?
Answer: Answer: no its not possible, we can increase it but not reduce them
Question: what is the use of elastic ip are they charged by aws?
Answer: Answer: These are  ipv4 address which are used to connect the instance from  internet, they are charged if the instances are not attached to it
Question: one of mys3is bucket is deleted but i need to restore is there any possible way?
Question: issue?
Question: is it possible to stop ards instance how can i do that?
Question: what is meant by parameter groups in rds and what is the use of it?
Answer: Answer: Since RDS is a managed service AWS offers a wide set of  parameter  in RDS as  parameter group which is modified as per requirement
Question: what is the use of tags and how they are useful?
Question: how can i rectify it?
Question: i dont want my aws account id to be exposed to users how can i avoid it?
Question: by default how many elastic ip address does aws offer?
Answer: Answer: 5 elastic ip per region
Question: you are enabled sticky session with elb what does it do with your instance?
Answer: Answer: Binds the user session with a specific instance Q67) Which type of load balancer makes routing decisions at either the transport layer or the Application layer and supports either EC2 or VPC. Answer: Classic Load Balancer
Question: which is virtual network interface that you can attach to an instance in a vpc?
Question: have selected ssh http https protocol why do we need to select ssh?
Question: security group how will these changes be effective?
Answer: Answer: Changes are automatically applied to windows instances
Question: loadbalancer and dns service comes under which type of cloud service?
Question: this?
Answer: Answer: Create a snapshot of the unencrypted volume (applying encryption parameters), copy  the. Snapshot and create a volume from the copied snaps hot Q73) Where does the user specify the maximum number of instances with the auto
Question: scaling commands?
Answer: Answer: Auto scaling Launch Config
Question: which are the types of ami provided by aws?
Question: a single instance what setting can you use?
Answer: Answer:   Sticky session
Question: when doi prefer to provisioned iops over the standard rds storage?
Question: db instance for read or write a operation along with to primary db instance?
Answer: Answer:  Primary db instance does not working . Q78) Which the AWS services will you use to the collect and the process e -commerce data for
Question: the nearby real time analysis?
Answer: Answer:   Good of Amazon DynamoDB. Q79) A company is deploying the new two -tier an web application in AWS. The company has to limit ed on staff and the requires high availability, and the application requires to complex Follow  Me : https://www.yo utube.com/c/SauravAgarwa l queries and table joins. Which configuration provides to the solution for companys
Question: requirements?
Answer: Answer:  An web application provide on Amazon DynamoDB solution.
Question: which the statement use to cases are suitable for amazon dynamo db?
Answer: Answer:The storing metadata for the Amazon S3 objects& The Running of relational joins and complex an updates. Q81) Your application has to the retrieve on data from your users mobile take every 5 minutes and then data is stored in the DynamoDB, later every day at the particular time the data is an extracted into S3 on a per user basis and then your application is later on used to visualize the data to user. You are the asked to the optimize  the architecture of the backend system can to
Question: lower cost what would you recommend do?
Question: mysql which is the best approaches to the meet these requirements?
Question: setup of following would you be prefer?
Answer: Answer:  The Replace the RDS instance with an 6 node Redshift cluster with take 96TB  of storage. Q84) Let  to Suppose you have an application where do you have to render images and also do
Question: some of general computing which service will be best fit your need?
Answer: Answer:Used on Application Load Balancer. Q85) How will change the instance give ty pe for the instances, which are the running in your
Question: applications tier and then using auto scaling where will you change it from areas?
Answer: Answer:  Changed to Auto Scaling launch configuration areas. Q86) You have an content management system running on the Am azon EC2 instance that is the approaching 100% CPU of utilization. Which option will be reduce load on the Amazon EC2
Question: instance?
Answer: Answer:  Let Create a load balancer, and Give register the Amazon EC2 instance with it.
Question: follow me https www you tubecomcsauravagarwalq87 what does the connection of draining do?
Answer: Answer:  The re -routes traffic from the instances which are to be updated (or) failed an health to check. Q88) When the instance is an unhealthy, it is do terminated and replaced with an new ones,
Question: which of the services does that?
Answer: Answer:  The survice make a fault tolerance.
Question: what are the lifecycle to hooks used for the auto scaling?
Answer: Answer:  They are used to the  put an additional taken wait time to the scale in or scale out events. Q90) An user has to setup an Auto Scaling group . Due to some issue the group has to failed for launch a single instance for the more than 24 hours. What will be happen to the Auto Scaling in
Question: the condition?
Answer: Answer:  The auto Scaling will be suspend to the scaling process. Q91) You have an the EC2 Securit y Group with a several running to EC2 instances. You changed to the Security of Group rules to allow the inbound traffic on a new port and protocol, and then the launched a several new instances in the same of Security Group.Such the new rules
Question: apply?
Question: region?
Answer: Answer:  May be the selected on Route 53 Record Sets. Q93) An customers wants to the captures all client connections to get information from his load balancers at an interval of 5 minutes only, which cal select option   should he choose for his
Question: application?
Answer: Answer:  The condition should be Enable to AWS CloudTrail for the loadbalancers.
Question: which of the services to you would not use to deploy an app?
Answer: Answer:  Lambda app not used on deploy.
Question: how do the elastic beanstalk can apply to updates?
Question: could be reason solution?
Question: accomplish this?
Answer: Answer:The monitoring on Amazon CloudWatch Q98) The organization that is currently using the consolidated billing has to recently acquired to another company that alr eady has a number of the AWS accounts. How could an Administrator to ensure that all the AWS accounts, from the both existing company and then acquired company, is
Question: billed to the single account?
Question: this scenario?
Question: use the aws credentials to access s3 bucket securely?
Question: you accomplish the using the aws services?
Question: setup?
Question: availability?
Question: disruptions during the ramp up traffic?
Question: instances which are architectural choices should you make?
Answer: Answer:  Deploy   to 3 EC2 instances in  one  of availability zone and 3 in another availability of zones and to use of Amazon Elastic is Load Balancer. Q106) You are the designing an application tha t a contains protected health information. Security and Then compliance requirements for your application mandate that all protected to health information in application use to encryption at rest and in the transit module. The application to uses  an three -tier architecture. where should data flows through the load balancers and is stored on the Amazon EBS volumes for the processing, and the results are stored in the Amazon S3 using  a AWS  SDK. Which of the options satisfy the security
Question: requirements?
Question: are will evenly distributed across to four web servers?
Question: capacity most of effectively?
Question: support these requirements?
Answer: Answer:The configure to the web application get authenticate end -users against the centralized access on the  management  system. Have a web application provisio n trusted to users STS tokens an entitling the download of the approved data directly from  a Amazon  S3. Q110) A Enterprise  customer is starting on their migration to the cloud, their main reason for the migrating is agility and they want to the make their internal Microsoft active directory available to  the many  applications running on AWS, this is so internal users for only have to remember one set of the credentials and as a central point of user take control for the leavers and joiners. How could they ma ke their  actions  the directory secures and the highly available with minimal on -premises on infrastructure changes in the most cost and the  time-
Question: efficient way?
Question: what is cloud computing?
Answer: Answer:Cloud computing means it provides services to access programs, application, storage, network, server over the internet through browser or client side application on your PC, Laptop, Mobile by the end user without installing, updating and maintaining  them.
Question: follow me https www youtube comcsauravagarwalq112 why we go for cloud computing?
Question: what are the deployment models using in cloud?
Answer: Answer:  Private Cloud  Public Cloud  Hybrid cloud  Community cloud 4
Question: explain cloud service models?
Question: what are the advantage of cloud computing?
Question: what is a ws?
Answer: Answer: Amazon web service is a secure cloud services platform offering compute, power, database, storage, content delivery and other functionality to help business scale and grow. AWS is fully on -demand AWS is Flexibility, availability and Scalability Follow  Me : https://www.yo utube.com/c/SauravAgarwa l AWS is Elasticity: scale up and scale down as needed.
Question: what is mean by region availability zone and edge location?
Question: how to access aws platform?
Question: what is ec 2whatarethebenefitsinec2?
Answer: Amazon Elastic compute cloud is a web service  that provides resizable compute capacity in the cloud.AWS EC2 provides scalable computing capacity in the AWS Cloud. These are the virtual servers also called as an instances. We can use the instances pay per use basis. Benefits:  Easier and Faster  Elastic and Scalable  High Availability  Cost-Effective
Question: what are the pricing models available in awsec2?
Question: what are the types using inawsec2?
Answer: Answer:  General Purpose  Compute Optimized  Memory optimized  Storage Optimized  Accelerated Computing (GPU Based)
Question: follow me https www youtube comcsauravagarwalq122 what is ami what are the types in ami?
Question: how to addressing awsec2 instances?
Question: what is security group?
Answer: Answer: AWS allows you to control traffic in and out of your instance through virtual firewall called Security groups. Security groups allow you to control traffic based on port, protocol and source/Destination.
Question: when your instance show retired state?
Question: start what is the reason for that and explain solution?