fabiochiu/medium-articles huggingface dataset:
Test Automation
By: Supuni Uthpala
Tags: Selenium, Test Automation

Test automation is the management and performance of test activities, to include the development and execution of test scripts so as to verify test requirements, using an automated test tool.

Types of Automated Testing

Code-driven — Unit Tests GUI(End to End Test Automation) API/Service Test Automation — Restful APIs, SOAP Mobile Test Automation — Android, iOS, Hybrid

Automated Tools

• Ranorex

• Selenium

• HP UFT( Unified Functional Test)

• Watir

TestComplete

Postman

Benefits of a Test Automation Framework

• Costs and efficiency

• Time economy

Quality

Reusability

Barriers of a Test Automation Framework

Need better tools and training

A lot of testing areas left uncovered

Lack of experience

Lack of clear goals

Selenium

Selenium is a suite of open source software testing automation tools used to validate web applications across different browsers and platforms. Multiple programming languages like Java, C#, Python can be used to create Selenium Test Scripts. Testing done using the Selenium testing tool is usually referred to as Selenium Testing.

Selenium Suite Of Tools

Selenium Integrated Development Environment (IDE)

Selenium Remote Control (RC)

WebDriver

Selenium Grid

Selenium IDE

In 2006, Shinya Kastani from Japan had donated his Selenium IDE prototype to Apache’s Selenium project. It was a Firefox plugin for faster creation of test cases.

IDE implemented a record and playback model wherein, test cases are created by recording the interactions which the user had with the web browser. These tests can then be played back any number of times. The advantage with Selenium IDE is that, tests recorded via the plugin can be exported in different programming languages like: Java, Ruby, Python etc.

Selenium RC

Selenium Core was the first tool in the Selenium project. But, Selenium Core hit a roadblock in terms of cross-domain testing because of the same-origin policy. The same-origin policy prohibits JavaScript code from accessing web elements that are hosted on a different domain compared to where the JavaScript was launched.

Selenium WebDriver

Selenium WebDriver, founded by Simon Stewart in 2006, was the first cross-platform testing framework that could control the browser from OS level. It provides a programming interface to create and execute test cases. WebDriver is an upgrade to RC because it is much faster. Unlike Selenium RC, WebDriver does not rely on JavaScript for Selenium Automation Testing. It controls the browser by directly communicating with it. Each browser has its own driver on which the application runs. The different WebDrivers are:

Firefox Driver (Gecko Driver)

Chrome Driver

Internet Explorer Driver

Opera Driver

Safari Driver and

HTM Unit Driver

Selenium Grid

Selenium Grid was developed by Patrick Lightbody and initially called HostedQA (initially a part of Selenium v1).This is used together with Selenium RC to run parallel tests across different machines and different browsers all at the same time. Parallel execution means running multiple tests at once.
Introduction to Test Automation
By: Bimali Wickramasinghe
Tags: Selenium, Software Testing, Test Automation Framework, Test Automation

Automation is the doing the repetitive tasks using the machines as tools reducing the human intervention. Humans are more error prone and less effective.

In definition given in Wikipedia, in software testing, test automation is the special software to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

Benefits of Test Automation

Test automations gives many benefits in various ways. The following are some benefits we can gain by automating testing.

Speedup test execution

Avoid human errors

Unattended execution — human effort is not necessary in execution

Test what manually impossible — ex: higher number of outputs

Improve efficiency of testing and cost

Improves team morale

More time for exploratory testing

Types of Automated Testing

Code-driven — unit tests/TDD

unit tests/TDD GUI(End to end test automation) — UI object property based, image recognition based

UI object property based, image recognition based API/ service test automation — restful APIs, SOAP

restful APIs, SOAP Mobile test automation — Android, IOS, Hybrid

Software Testing Pyramids

Tools and Technologies for Automated Testing

GUI (for Web) — Unified Functional Testing, TestComplete, Selenium, Sikuli, Protractor, Watir

Unified Functional Testing, TestComplete, Selenium, Sikuli, Protractor, Watir Mobile — Appium (FOSS), Calabash, DeviceAnywhere, Perfecto

Appium (FOSS), Calabash, DeviceAnywhere, Perfecto GUI (for Windows) — TestComplete, Unified Functional Testing, Ranorex, Sikuli

TestComplete, Unified Functional Testing, Ranorex, Sikuli API — Unified Functional Testing, SoapUI, Rest-assured, POSTMAN, FitNesse

Test Automation Frameworks for GUI Applications

Data Driven Framework — Data -> Action Script -> Result

Data -> Action Script -> Result Keyword Driven Framework — Keywords -> Driver Script -> Result

Keywords -> Driver Script -> Result Page Object Design — Page Objects -> Test Script -> Result

Page Objects -> Test Script -> Result ATDD (Acceptance Test Driven Development)

Why Automation Framworks?

Framework is supporting structure which something can be built.

Frameworks,

Increase reusability

Increase maintainability

Reduce complexity

Improve reporting

Improve error handling

Improve productivity

To gain the maximum ROI, a person must have a better understanding about the testing pyramid (scope of automation), right automation tools, choose appropriate framework, scripting standards, measure metrics and team.

Introduction to Automation Frameworks Using Selenium Web Driver

Components of Selenium

Radio buttons, drop downs, checkboxes, text fields, buttons, labels, links and any kind of element in the web page are web element locators. Selenium is an open source tool used for test automation in web applications. Selenium is a package with many options. Selenium is widely used for test automations.

Refferences :
Test Automation
By: Chamodi Jayodya
Tags: Selenium, Test Automation

In our degree program, there is a compulsory course module we follow in our second year. That is Software Verification and Validation. As a part of this course module, we had a session about Test Automation conducted by an associate conduct specialist. So let’s see about test automation and tools which are useful for software engineers.

As the Wikipedia definition, In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

First, he mentioned about black box testing which is a software testing method with functionalities of software applications and tested without knowing internal code structure, implementation details, and internal paths. Black Box Testing mainly focuses on the input and output of software applications and it is entirely based on software requirements and specifications. There are several benefits of test automation.

· Speedup test execution

· Avoid human errors

· Unattended execution

· Test what manually impossible

· Improve the efficiency of testing and cost

· Improve team morale

· More time for exploratory testing

Because of these benefits automation becomes much easier and efficient. There are different types of automated testing. They are;

Code driven — Unit tests/TDD

GUI (End to End Test Automation) — UI object property based / Image recognition based

API/Service Test Automation — Restful APIs, SOAP

Mobile Test Automation — Android, IOS, Hybrid

As we were told having the knowledge of the Testing pyramid is a key point in software engineering.

Testing Pyramid

Knowing the testing pyramid helps to divide time and effort in testing as if Unit testing needs the biggest coverage. And knowing the exact route makes finding the problem a lot easier. And also it needs to choose the correct automation tool in each testing stage.

He mentioned some of the industrial trending tools & technologies that are used in test automation.
Test automation: 3 things you need to know
Tags: Automation Testing, Process Improvement, Process Automation, Automation, Test Automation

Test automation — using automation tools to execute test case suites — delivers numerous benefits, including greater time- and cost-efficiency, the ability to run tests unattended/overnight, and a lower risk of integration and production issues.

Particularly well suited to automation are test cases that are:

New or modified functionalities

Business critical

Repetitious

Tedious for humans to perform

Performance sensitive

Time consuming

It’s important to recognize that automation can’t eliminate all manual testing because automation is for testing functionality. You still want users doing hands-on testing to ensure the usability of the application.

If your organization is considering automating parts of your testing processes, here are three things to keep in mind.

1. Cost and time requirements will be lower than you think.

Many teams don’t implement test automation because they believe that it takes too much time or too many resources. While this may have been true in the earlier days of test automation, it is no longer the case with modern tools and cloud services. For example, most tools can record user interactions and then allow developers and testers to modify the results for more dynamic testing. Also, most automated build tools can now incorporate many test engines into the build process, catching issues before they are deployed, and can impact live production use.

2. There are no static inputs in the real world.

Teams that do automation testing often use static, hard-coded parameters rather than dynamic parameters. This not only misses mimicking real-world use cases; it also will not properly reflect performance and scalability metrics where caching is in use. Dynamic parameters can either be randomly generated using standard scripting languages or driven from prepared input files. The parameters also need to be as realistic as possible. The use of nonsensical values to populate text fields and simple sequential numbers for complex number fields such as phone or amount can miss validating even the minimal edge cases

It is also important to have in place a procedure to reset databases to pre-test conditions for consistent re-testing.

To view the 3rd thing you need to know, and the bottom line, read through our full article here: https://www.logic2020.com/insight/tactical/test-automation-three-things-to-know?utm_source=social&utm_medium=medium&utm_campaign=Test_Automation
Karate - Cucumber Reports integration in Jenkins
By: Niron Rasanjana
Tags: Reporting, Jenkins, Karatê, Test Automation, Api Testing

In this post I will go through steps to integrate Cucumber reports in Jenkins for Karate tests. If you have not performed karate — Jenkins integration, please go through my previous post.

I. Install Cucumber Reports in Jenkins

Go to Jenkins Dashboard ->Manage Jenkins ->Manage plugins ->Available -> Cucumber reports -> Install

II. Add and Configure Plugin Once Installed

Go to the target Maven project

2. Click on Configure

3. Go down to Post-build Actions

Post-build Actions-> Add Post-build Actions -> Cucumber Reports -> Advanced

4. Set the following values

Json Report Path : target/surefire-reports

File Include pattern : *.json

III. Add Cucumber reporting dependency in to pom.xml

<dependency>

<groupId>net.masterthought</groupId>

<artifactId>cucumber-reporting</artifactId>

<version>3.8.0</version>

<scope>test</scope>

</dependency>

IV. Create a Parallel Runner class that generates the reports

import com.intuit.karate.KarateOptions;

import com.intuit.karate.Results;

import com.intuit.karate.Runner;

import net.masterthought.cucumber.Configuration;

import net.masterthought.cucumber.ReportBuilder;

import org.apache.commons.io.FileUtils;

import org.junit.Test; import java.io.File;

import java.util.ArrayList;

import java.util.Collection;

import java.util.List; import static junit.framework.TestCase.assertTrue;

public class TestParallel {



@Test

public void testParallel() {

System.setProperty("mock.env", "karateTesting"); // ensure reset if other tests (e.g. mock) had set env in CI

Results results = Runner.parallel(getClass(), 5);

TestParallel.generateReport(results.getReportDir());

assertTrue(results.getErrorMessages(), results.getFailCount() == 0);} @KarateOptions (tags = {"~ @ignore "})public class TestParallel {public void testParallel() {System.setProperty("mock.env", "karateTesting"); // ensure reset if other tests (e.g. mock) had set env in CIResults results = Runner.parallel(getClass(), 5);TestParallel.generateReport(results.getReportDir());assertTrue(results.getErrorMessages(), results.getFailCount() == 0);} public static void generateReport(String karateOutputPath) {

Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[] {"json"}, true);

List<String> jsonPaths = new ArrayList(jsonFiles.size());

jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));

Configuration config = new Configuration(new File("target"), "karateTesting");

ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);

reportBuilder.generateReports();

}

}

Your Karate — Cucumber reports integration is good to go. Let’s Build Now. You can view generated Cucumber reports in the Project directory.

Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.

To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇
The Role of QA in DevOps
Tags: Ci Cd Pipeline, Cicd Tools, QA, DevOps

This article will explore the role of QA in DevOps, highlighting the place DevOps testers hold in the ecosystem.

Let’s begin with defining some of the relevant terms:

DevOps: DevOps applies Agile practices to QA and Ops teams, thus streamlining the build, validation, deployment, and development of software. It seeks to standardise development environments and improve predictability, efficiency, and security in the software development lifecycle.

Refer to this detailed article on DevOps.

CI/CD: CI or Continuous Integration is an engineering practice in which team members integrate their code at a very high frequency. CD or Continuous Delivery is the practice of ensuring that code is always in a deployable state.

Refer to this detailed article on CI/CD for more information.

Continuous Testing: Continuous Testing refers to a set of automated tests that are executed every time a new code build is pushed to the repository.

The Role of QA in DevOps

The QA as a Consultant

DevOps is all about shifting individual and collective mindsets in more collaborative directions.

However, to execute this shift, devs will require some training. With devs taking on more testing, QAs will transition to a more consultative role, helping devs create better tests and become better at screening for quality.

QAs would also be responsible for helping devs gain a better understanding of testing philosophies and processes. This fundamental shift in the technical approach cannot be performed without serious engagement and input from QA departments.

The QA as a Strategist

As a strategist, the QA engineer makes high-level decisions about what standards the software should meet in its final form. Since QAs must have intimate knowledge of how end-users think and interact with software, they are best placed to advise dev teams on what features to include, what common bugs to look out for in unit tests, and what kind of users the software would be targeted towards.

As strategists, QAs are also responsible for selecting the most suitable tools and frameworks for manual testing and automation testing. Remember, successful DevOps implementation relies heavily on automation. It is of utmost importance to use the right frameworks, established in the right toolchains that will automate large sections of the development and deployment pipeline.

Run manual and automated tests on real browsers and devices. Start running tests on a real device cloud. Run parallel tests on a Cloud Selenium Grid to get faster results without compromising on accuracy. Leverage integrations with the most popular and effective CI/CD tools to execute DevOps-style development. Detect bugs before users do by testing software in real user conditions.
Optimized processes using a combination of DevOps and QA
Tags: Agile Methodology, Software Testing, QA

Understand how to overcome challenges with optimized processes using a combination of DevOps and QA

There is a need for faster yet very high-quality testing which can be tough to achieve without the right approach. Here are some best practices to overcome challenges with optimized processes using a combination of DevOps and QA, that can help you achieve the results you want to achieve without compromise in quality.

1. Continuous Testing is the Key

Continuous Testing is one of the pillars of successful DevOps implementation. Continuous testing, continuous integration, continuous delivery, and DevOps are the four essential steps to developing and testing software at a rapid pace.

Continuous testing can be the most important factor in determining the success and timely completion of a project and its implementation can help QA teams achieve the goal of faster time to market.

2.Implement Automation to the best possible Extent

The required rates of automation to make continuous testing success is at least 85% or higher, however, studies show that the average rate is 20%. Very far from success.

This is in spite of the fact that the Agile methodology insists on automation of the testing areas which require little or no human intelligence to be executed, such as repeated test steps with different data sets. These instances should be automated to bring down the time required for testing. Use advanced test automation tools to help you to achieve this. This cannot be denied with frequent changes in code comes the requirement of loads and loads of testing.

3. Go for Rapid Iterations

When working in a DevOps environment, the objective is to successfully implement continuous integrations and development, this approach of iterations becomes very essential.

It is only with rapid iterations that teams can achieve the goal of providing users with updates that create an impact. Every developer understands that only quality will make the users feel the software is worth.

4.Analysis of Test Performance through Key Metrics

Like in any project, no one can be really sure whether the project is heading in the right direction or not, project managers need to study the test metrics. It helps understand the areas where test executions are failing, the time taken to execute test cases, the time spent by QA resources in the process.

A thorough understanding of test performance through analysis of key metrics will provide the data required to formulating strategies and improvising the test approach.

Conclusion:

To meet the changing needs of a very highly competitive market which focuses on the quality of the software and simultaneously wants faster updates and quick releases DevOps and QA teams need to work in sync for better results.
Why QA Automation is Important and Quality Engineering road Map
By: Irosha Dewmini
Tags: Why Automation, Qa Testing, Qa Road Map, QA

Bored Doing Manual testing? Tired of same test case writing and same Bug reporting? May be yes!🥱 But can we automate every thing? Big No. Both manual and automation testing are really important . Here we will walk through Importance of QA Automation and What skills you should learn to be an expert in Quality Assurance Engineering. 😍

Importance of QA Automation ?

Manual vs Automation

With the development of technology every thing has been automated to make the human activities easy and fast.

Ex: Automated school buses and vehicles, Automated doggy doors, Robotic gas pumps, coffee makers, and etc. are few examples in real life. 🚌

Same way in Quality Assurance engineering we can use Automation tools to do testing activities. There are lot of advantages of QA automation- Here are few of them.

Save money 🤑 and time ⏳: manual testing is time and cost consuming Increase Efficiency and effectiveness and also speed of test execution and test coverage. Avoid repeating same thing and higher test coverage Reduce business expenses Lower maintenance cost Difficult to test multilingual sites manually. Can run automated tests over night.( no need human presence)

Automation will never allow the same bug to bite you twice.

But in automation we can’t automate all the things. Then?

Which test cases to automate?

Test cases that are executed repeatedly.

2. Test cases that which are time consuming

3. Test cases that are very difficult to perform manually and that are very tedious.

4. Test cases which are in high risks and business critical.

Which test cases not to automate?
Testing tricks
Tags: Mobile App Development, QA, Mobile Apps, Tricks, Testing

3. Other cases

There can be a failure if you give an access in an application to use something (camera or files, for example) on Android, then minimize an app and turn the access off in application system settings.

If you press different buttons fast or at the same time there is a possibility of an application failing. It needs to block it: add loading indicators, block UI after press, etc. not to cause a fail by the user.

After device rotation an application can work incorrectly or can not clear enough draw interface. In this case, if specification doesn’t require landscape mode, you can block this option on phones, at least (because of uncomfortable usage and small screen). Many users often use tablets with landscape mode, so for the tablets you have to do the screen redraw correctly by the orientation changing.

Also don’t forget about incorrect data. It needs to avoid incorrect data: do regex or use specific keyboard type. For example, if it requires to enter pages’ range, you have to forbid letters’ entering (doesn’t matter how but it has to work steadily and correctly). It’s worth to remember that even an empty line, point or comma in any field can cause incorrect work or failure.

Summing Up

It’s hardly ever that users will do this staff but you need to be aware: work on cases when application crashes (closes with “application is failed” — Android, just closes — iOS). Failure or crash (use what you like) is a critical bug. Below there is a dialogue box when it’s application crashes (concrete app in inverted commas).

In the released app there has no be such a failure. The user won’t like it, rate an application badly and will delete an app. The others will see bad rates and won’t install an app. It will cause the loss of users.

It’s ̶h̶a̶r̶d̶ impossible to be aware of all the situations, especially when a user changes something in system settings, so you have to be responsible for testing and do not miss crucial moments. The application quality is vital today.

P. S. Screenshots were taken from Foamline Transport application from our portfolio list: http://bytepace.com/foam-transport.

Thanks for reading! And join our website for more articles: http://bytepace.com.
[Eden Diary_TD] QA by Kevin Yu
Tags: Blockchain, QA, Edenchain, Bitcoin, Cryptocurrency

QA

by Kevin Yu

About quality

Good quality can be defined as the successful completion of a project developed without a bug while meeting the needs of prospective users within a planned schedule and budget. “QA” stands for quality assurance. It refers to the task of performing various tests and inspections prior to the launch of a product in order to ensure that the service or product has achieved its objectives and can be used without problem.

In testing, both normal and abnormal use cases are carried out to identify defects. This ‘Detection’ activity is basic and ongoing. The goal of a QA effort varies from finding bugs to finding out whether a user is achieving what he or she intended, identifying potential serious issues in advance, and verifying whether any subsequent modifications were properly applied. Continuous ‘Detection’ activities can minimize problems and risks in advance and can lead to the good user experience. Most efficient QA activities have a significant impact on the overall schedule and development costs of the project. The completion of the QA process is an integral part of the completion of the entire project.

Why does a bug occur?

Miscommunication or No communication: Poor communication can cause a bug. Therefore, clear communication must be recorded on a document.

Software complexity: Whether it could be a flawed design or just a feature of a complex system, this requires more time to analyze and requires stress testing the complex modules.

Programming errors: Nobody can develop a 100% error-free program.

Changing requirements: The most common cause of issues are due to changes in requirements; simple changes in requirements can affect an entire whole system.

Without QA…

It is difficult for developers to verify everything objectively. After the occurrence of a bug, it may often difficult to organically reproduce and detect its actual source. This makes schedule management difficult because of the impact on development and associated modification work. Note, however, that the impact does depend on the situation and the scale of a project. It is, however, a burden on developers to carry out a project in a situation where time is limited. This can actually reduce the efficiency of the development. Therefore, testing of technical issues and functional risks through a separate testing team can be a great complement to the development environment and to overall project management
What’s the biggest influencer for tool choice?
By: Ministry Of Testing
Tags: QA, Software Testing, Software Development, Testing, DevOps

Ministry of Testing’s Weekly Software Testing Newsletter

This newsletter is kindly supported by Vornex!

Temporal testing is critical for date sensitive business rules such as insurance rates, billing, and enrollment. Efficient time travel testing can increase your test coverage, accelerate development, and improve software quality. Read our whitepaper to learn the 5 ways to time travel software!

All Things Ministry of Testing

The latest news, along with recent Dojo additions, events and interesting club posts.

This newsletter is also kindly supported by Wonderproxy!

From our versatile testing solutions to our worldwide network of proxy servers, WonderProxy is built for localization testing. Squash those bugs and ship that next release.

Community News & Podcasts

Posts from our blog & podcast feeds — you can always check out the feeds if you want more.

99-Minute Workshops — October

Find out what time the workshops are happening in your time zone by clicking on a workshop.

Business

Posts selected from our business feed

Upcoming MoT Events

Things You Might Have Missed

Articles and posts on Security, Accessibility and Automation from around the web.

Web Accessibility: Definitions and Acronyms — Initially, learning about web accessibility, its policies, and its guidelines can seem daunting, but taking the time to learn and understand the basics is well worth the effort.

— Initially, learning about web accessibility, its policies, and its guidelines can seem daunting, but taking the time to learn and understand the basics is well worth the effort. Chrome changes how its cache system works to improve privacy — Chrome 86, launched this week, introduces new privacy-focused “cache partitioning” mechanism.

— Chrome 86, launched this week, introduces new privacy-focused “cache partitioning” mechanism. Fitbit gallery can be used to distribute malicious apps — A security researcher discovered that malicious apps for Fitbit devices can be uploaded to the legitimate Fitbit domain and users can install them from private links.

Meetups
Optimizing for happiness
By: Craig Mayhew
Tags: UX, QA, Software Development, Serverless, DevOps

We were a little late to the cloud party, but we are now well and truly caught up. 3 years back we began a transformation at CRC that we quickly realised would have no end date. Our first step was a jump from our own hardware and into the cloud then we took a fresh look at buying services to minimize what we build. Now we look at what else and what next.

Cloud coupled with infrastructure as code brought stability and flexibility with a significant reduction in ops time and therefore cost. We suddenly had new options on the table. We could for the first time scale vertically. We had quick and low effort recovery from failures. We could now consider short lived infrastructure as a possible answer to challenges.

Now able to move out of continually reacting, we could finally reflect on our process and ourselves.

We had a QA team at this point in time, and as you might imagine, this wasn’t working. The incentives for a QA team can quickly result in bug hunting rising above all other concerns. Unfortunately, the QA team would feel pressure when bugs made it to production, further fuelling the desire to find bugs above any other activity. The devs however will be under pressure to deliver a feature and move onto the next. These two teams quickly became locked in a tennis match when they should have been working together to complete a sprint. We recognised this needed to change. Instead of a QA team, we moved to a QA coach model. The developers write the tests and the coach helps them to write better automated tests through workshops and 1:1 coaching. The cycle then becomes one of continual iterating improvement in code quality. This enables our QA coach to continually soak up the best in the industry knowledge and distribute that knowledge to the team. Everyone wins.

The story was similar in DevOps. We had people we called “DevOps” who were looking after the infrastructure and trying to keep everything online above all else. The devs wanting to introduce change, was therefore heavily controlled to prevent expected downtime. We turned this into a devops coaching role, and it is fair to say this takes a long time to implement. Our devops coaches upskilled our developers on AWS, infrastructure as code and eventually serverless. We’ve also introduced some new roles with the same model, including an agile and also a UX coach.

The developers now have huge support to write great tests, write great infrastructure, write great code and iterate based on user feedback. The future entries in the blog will be from the team. We are setup for success, and I look forward to reading about it!
Prophet any Stock in Seconds
By: Winston Robson
Tags: Prophet, Open Source, Stocks, Data Science, Time Series Forecasting

Prophet any Stock in Seconds

Install yfinance

With the Yahoo! Finance market data downloader (yfinance), we can pull historical data on virtually any stock with a single line of code.

You can install yfinance with pip;

pip install yfinance

From there, simply import the library, and pull a .Ticker() .

Pull & Prep Data

Let’s do the New York Times Company;

That outputs a yfinance.Ticker object, which holds historical $NYT data accessible with .history() . Setting period=’max’ will return all data;

Prophet expects input data to have 2 columns, ds and y , so let’s just copy the historical dates ( hist.index ) and adjusted closing prices ( hist[‘Close’] ) to a new DataFrame.

Forecast with Prophet

Now it’s time to make a Prophet model and fit it with out data;

Finally, let’s define a future and .predict() it;

The New York Times Company ($NYT)10 year forecast

Fin

Thank you for reading! Please feel free to respond with any questions.

Find this story’s notebook here: intro_to_prophet/prophet_yfinance.ipynb

Run the notebook free in Colab: colab/prophet_yfinance.ipynb

You can help build yfinance on GitHub.

Continued Reading

References

Aroussi, Ran. “Reliably Download Historical Market Data from Yahoo! Finance with Python.” Ran Aroussi (Official Website), 17 Apr. 2019, aroussi.com/post/python-yahoo-finance.
Hourly Electricity Price Forecasting using Long Short-Term Memory Neural Networks
By: Matteo Bonanomi
Tags: Electricity, Price Forecast, Machine Learning, Time Series Forecasting, Neural Networks

Technical background

European electricity spot market (also called day-ahead market) have become more and more fluctuating as the renewable energy share became larger. Renewable energy, often bidded at very competitive prices in the market, have a huge impact on regional electricity price and depends on non-deterministic factors, such as weather conditions and their (often unreliable) forecasts. For these reasons, renewable energy forecasting and electricity price forecasting are a complex matter, commonly addressed in scientific literature.

Aim of the Work

The analysis I performed compares a trivial baseline model (called persistence model) to an advanced time series forecasting technique: Long Short-Term Memory Neural Networks. This complex algorithm is widely studied when time series forecasting is concerned. Its black-box behavior and difficulty to interpret are two main drawbacks, yet its performance can beat traditional ARIMA models when complex problems are addressed and large datasets are available. The aim of my work is to develop a simple and modular code to train an LSTM using an open electricity price dataset. Performance of this model are estimated varying the time forecast horizon, so that its benefits compared to a simple baseline model can be clarified.

Dataset and baseline model

Nord Pool website is a huge open data source when Northern European Electricity Market is concerned. You can download the dataset I used HERE. This analysis is performed on hourly price data, but the code I developed is quite flexible and modular and can be reused and tuned for any kind of analysis.

The baseline to beat for the LSTM model is quite simple: a persistence model. This means that the forecasted price at a certain hour is going to be the last price I observed. If I have access to real-time data, the persistence model will predict at hour t the same price I observed at hour t-1. In many cases, one might be interested in predicting electricity price with a few hours in advance. For instance, given the last known price at time t, I would like to know the price at t+2h, t+3h, t+4h and so on. For this reason, the persistence model might be quite effective with forecast quite close to the last available data, especially with time series that are not so fluctuating (electricity price is not cypto!). Later on, I will show the performance of this kind of model as a function of the last available hourly price data. It is easy to understand model performance are going to be decent only when we are very close to the last known hour and they will decay quite suddenly. A new LSTM must be able to beat this baseline, especially for a daily forecast: 1–24h from the actual time.
Time Series prediction using Recurrent Neural Networks
By: Marianne Benkamoun
Tags: Machine Learning, Time Series Forecasting, Rnn, Stock Market

2. Timeseries prediction with only one variable using LSTM

Stock prices follow a Random walk model. Given the way that the random walk is constructed, we can expect that the best prediction we could make would be to use the observation at the previous time step as what will happen in the next time step.

Let’s first import the required libraries and have a look at our data.

Here is what our data looks like:

The value that I want to predict is the fourth column, ‘Close’. Let’s define Y as this column and have a look at its trend.

We will use shifted versions of the column Y as independent variables, that is to say use 3 delays of Y as inputs to predict the output of our data.

At this point, it is very important to standardize our data. We choose the MinMaxScaler from sklearn.preprocessing library that will scale X between -1 and 1 and we do the same thing for Y.

LSTM needs 3 dimensional input so we have to reshape the X input into 3 dimensions.

Let’s now define training and test sets for our model!

We can now construct our model.

For the first layer of the model, I chose 10 units, hyperbolic tangent for the activation. Here we have an input shape of (1,3) because we have 3 delays for the Nasdaq index (the X1, X2 and X2 that we defined before). We use hard sigmoid for reccurent activation.

Then we need to had and output layer to it. We use only one output for our model.

Once we’ve created our neural network, then we must compile it. We use mean absolute error to assess our model and then we fit our model to the training set and launch the prediction !

You can tune your own parameters, check the documentation to gain more knowledge!

Let’s now have a visualization comparing the prediction versus the test set to see how our model performed!

Not too bad right ? Try to tune the model’s parameters to see your output.

Let’s now add time indexes and scale back to the original scale!

2. Timeseries prediction with multiple variables using LSTM

In this section, I will use the columns ‘High’ and ‘Low’ as independent variable X to train my model, and the same column, ‘Close’ as the dependent variable Y. Let’s give it a try!

First, let’s vizualize the High, Low, and Close columns.

Let’s do the exact same process that we did before for the time series with one variable : splitting into test and train set, fitting our model and vizualize it.

Let’s now scale back our data and had a time index to have a clearer view of our predictions.

And.. that’s it ! We successfully used RNN and LSTM to predict the closing stock price of NASDAQ, using the last 3 trailing days as independent variables, and then high and low stock prices as independent variables.

I hope you found this article useful, see you soon !
Beginners Guide: VPS Setup for Bug Bounty Recon Automation
By: Ranjan Kumar
Tags: Bug Bounty, Infosec, Security, Penetration Testing

Hello, All. My name is Ranjan. I am a final year CS undergrad and a part-time bug bounty hunter. Due to college, I don’t get enough time for bug bounty hunting. So, I decide to automate some of my tasks, and for that, I was searching for a VPS(Virtual Private Server).

So I found out there is a lot of VPS in the market like Digital Ocean VPS, Linode, and google cloud. But I liked Google cloud because of its free tier offer.

I will be using Google cloud in this writeup and will provide you step by step procedure for setting up your VPS.

Let’s get start:

For new customer, google cloud provide 300$ free credit which you can utilize up to three months. You need to add a billing account in the google cloud.

You can claim it here:

https://cloud.google.com/free

Step 1: Select your country.

Step 2: Fill in all the necessary details and your card details as well.

Now, you have to verify your payment by providing a pic of your credit card or giving a screenshot of your bank account statement.

Verification hardly takes 2–4 hrs. After verification, you are good to go.

Now go to the console and select Compute Engine>VM Instances in the left-hand menu bar.

After this, google will do some initialization tasks, including setting up an initial project called My First Project, so you don’t have to take care of that.

When the platform initialization task is finished, you can create your first VM by clicking on the Create instance button.
The Most Used MITRE ATT&CK Technique: T1055 Process Injection
By: Suleyman Ozarslan
Tags: Hacking, Infosec, Red Team, Cybersecurity, Mitre Attack

Our research has shown that the most prevalent MITRE ATT&CK technique used by adversaries in their malware was T1055 Process Injection. Adversaries emphasize an enhanced level of stealth, persistence, and privilege in their sophisticated cyber attacks. As a mechanism capable of providing these functions, it is not surprising that Process Injection is the most frequently used technique.

By listing the running procedures and filtering out legitimate ones that are part of the operating system or installed applications, it is simple to identify malware processes. However, if the malware is able to encapsulate its malicious code inside a legitimate process, it will hide on the infected system. In fact, process injection is a “old but gold” method consisting of running arbitrary code within another process’s address space. As a consequence, this technique allows access to the memory, device, and network resources of the target process.

The technique offers three major advantages for adversaries on this account:

Under a legitimate method, executing code can evade security controls. Since the legitimate process is whitelisted, it camouflages the malicious code to avoid detection.

Since the malicious code is executed within the memory space of the legitimate process, it can also evade disk forensics.

If the target process has elevated privileges, this technique will enable privilege escalation. For example, if the target process has access to network resources, the malicious code can communicate legitimately over the Internet and with other computers on the same network.

This technique will allow privilege escalation if the target process has elevated privileges. For instance, if the target process has access to network resources, the malicious code will legitimately communicate with other computers on the same network and over the Internet.

Targeted Processes

Since custom processes quickly detected by security controls, threat actors use common Windows processes such as:

Built-in native Windows processes: AppLaunch.exe, arp.exe, cmd.exe, csc.exe, cvtres.exe, dllhost.exe, explorer.exe, msbuild.exe, PowerShell.exe, RegAsm.exe, RegSvcs.exe, regsvr32.exe, rundll32.exe, services.exe, svchost.exe, vbc.exe,

Common software processes: chrome.exe, firefox.exe, ieuser.exe, iexplore.exe, msinm.exe, opera.exe, and outlook.exe.

Sub-techniques of the Process Injection Technique

There are 14 sub-techniques of Process Injection:

T1055.001 Dynamic-link Library Injection

T1055.002 Portable Executable Injection

T1055.003 Thread Execution Hijacking

T1055.004 Asynchronous Procedure Call

T1055.005 Thread Local Storage

T1055.008 Ptrace System Calls

T1055.009 Proc Memory

T1055.011 Extra Window Memory Injection

T1055.012 Process Hollowing

T1055.013 Process Doppelgänging

T1055.014 VDSO Hijacking

Read our blog for a more in-depth look at the MITRE ATT&CK T1055 Process Injection technique.
Building Future-Proof Cybersecurity Education Programs with New York University’s Dr. Nasir Memon — AccessCyber
By: Gotham Sharma
Tags: Tech, Hacking, Infosec, Accesscyber, Cybersecurity

Building Future-Proof Cybersecurity Education Programs with New York University’s Dr. Nasir Memon — AccessCyber

Professor of Computer Science and Engineering, Co-Founder of the NYU Center for Cyber Security, Founder of the Osiris Lab & CSAW; In more ways than one, Dr. Nasir Memon (NYU Tandon) is a pioneer in the field of cybersecurity education. On this episode of the AccessCyber Podcast, Dr. Memon joins us to chat about all things infosec, including

How the security industry has changed over the last 2 decades

Defining cybersecurity from an academic lens

The origins of information security as an academic discipline, and how it relates to business, mathematics, and computer science

What does it actually mean for a system to ‘be secure‘

Models for how industry and academia can work together to better serve students

What students should and shouldn’t expect from the college experience

Studying algorithms and learning how to think

Foundational learning versus role-based education

Disruption in higher education in the midst of a pandemic

Working with Phil Venables, Splunk, Qualys, ZScaler, and others

Listen to the full episode.

Originally published at https://www.accesscyber.org on November 23, 2020.
How the simple flaw leads to leakage of all questions on the school examination website.
By: Binamra Pandey
Tags: Infosec, Hacking, Vulnerability, Flaws, Cybersecurity

How the simple flaw leads to leakage of all questions on the school examination website. Binamra Pandey Follow Jul 15 · 2 min read

After the Covid blew up, everything has moved online. School started to teach online to its student and even decided to take exams online. My school also decided to take exams online using certain examination website.

On the surface website works like this, First school uploads all the questions according to subjects then on the exam day at specific time we can start the exam. It was a multiple-choice question, questions would be displayed randomly to every student and every question has a timer set on. So it would be harder for students to cheat.

I started my burp and started to capture the request and after some time I found out the following JSON request.

{

“examEndTime” : “2021–07–11T09:10:00”,

“examStartTime” : “2021–07–11T09:00:00”

}

This JSON request represent exam start or end time. Before “T” in that request, it's a date to start, and after “T” it’s time to start. So basically I guess T=time to start or end. So I changed this time to the present date and time hoping I will be able to see questions but I failed.

So now I decided to check its response, so I send this repeater and forwarded the request to see its response and I got the following response to that request.

So now I knew what to do, I captured the initial request and set burp to capture its response also and when I get the response I changed “isReady” to true and BOOM I was able to get the question. But the website won’t let me see all the questions, I have to submit answer to the first shown question to move out to the next question.

So I thought of something, I just select any random answer and click on “submit” but when I get this request to burp, I Dropped the request and I thought it would give me a Burp error page, but it didn’t instead it moved me to next question. So I can just submit any random answer and drop the request and I was able to see all the questions. Since I dropped that submit request, so the website thinks I haven’t submitted the answer, so when it's time to give the exam I can give the correct answer to every question.

This website was vulnerable to a flaw called “Response Manipulation”. By manipulating the response I was able to see questions before the time of the exam.

After some time I reported this to that website after completion of my exam

Thank You.