teleren/devto huggingface dataset:
Best Practices When Designing AWS Architecture: Security and Operational Excellence
Tags: aws, security, operations, cloud
Published: 2023-04-25 05:43:00

The AWS Well-Architected Framework helps you understand the benefits and risks of decisions you make while building workloads on AWS. By using the Framework you will learn operational and architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable workloads in the cloud. It provides a way to consistently measure your operations and architectures against best practices and identify areas for improvement.

The framework is based on six pillars: 

- Operational Excellence
- Security
- Reliability
- Performance Efficiency
- Cost Optimization
- Sustainability

This post focuses on the Operational Excellence and on the Security pillars.

Operational Excellence is the first pillar of AWS Well-Architected Framework. It includes the ability to support the development and run workloads effectively while gaining insight into operations and continuously improving processes and procedures to deliver business value.

By adopting the practices in this pillar you can build architectures that provide insight to their status, are enabled for effective and efficient operation and event response, and can continue to improve and support your business goals.

This pillar provides an overview of design principles, best practices, and questions. 

The Security pillar encompasses the ability to protect data, systems, and assets to take advantage of cloud technologies to improve your security. The pillar allows making use of cloud technologies to predict, prevent and respond to any threats as well as enforce privacy, data integrity, guard assets, and enhance detection of security events within a software environment.

By adopting the practices in this pillar you can build architectures that protect your data and systems, control access, and respond automatically to security events.

This pillar provides an overview of design principles, best practices, and questions. 

If you have any questions, feel free to reach out.
AWS Rekognition
Tags: aws, cloud, cloudcomputing, machinelearning
Published: 2023-04-25 05:41:46

Amazon Rekognition is a cloud-based computer vision service provided by Amazon Web Services (AWS). It allows developers to add advanced image and video analysis to their applications without requiring any expertise in computer vision or deep learning.

With Rekognition, you can identify objects, people, text, and activities within images and videos. Some of the specific features include:

* Object and scene detection: Rekognition can detect and label objects and scenes within images and videos, such as cars, buildings, or landscapes.
* Facial analysis: Rekognition can detect and analyze faces within images and videos, including emotions, age, and gender.
* Celebrity recognition: Rekognition can identify celebrities within images and videos, and provide additional information such as their occupation or other related images.
* Text detection: Rekognition can detect and recognize text within images and videos, such as signs, labels, or captions.
* Content moderation: Rekognition can help identify inappropriate or unwanted content within images and videos, such as violence or nudity.

Rekognition can be used for a variety of use cases, such as security and surveillance, media analysis, customer engagement, and content moderation. It provides a scalable and cost-effective solution for developers to add advanced image and video analysis capabilities to their applications.

## How to use AWS Rekognition?
To use AWS Rekognition, you can follow these steps:

1. Create an AWS account: If you don't have one already, you'll need to create an AWS account to use Rekognition.

2. Set up your IAM user and permissions: You'll need to set up an Identity and Access Management (IAM) user and permissions to access Rekognition. This user will need permissions to access the Rekognition service.

3. Upload your images or videos: You can upload your images or videos to Amazon S3 or directly to Rekognition.

4. Use Rekognition to analyze your images or videos: You can use the Rekognition API or console to analyze your images or videos. Some of the common use cases include object and scene detection, facial analysis, celebrity recognition, text detection, and content moderation.

5. Use the results: Once Rekognition has analyzed your images or videos, you can use the results in your application. For example, you might use facial analysis to identify customers in a retail store, or use object detection to identify items in a warehouse.

AWS provides detailed documentation on how to use Rekognition, including sample code and tutorials. The Rekognition console also provides a user-friendly interface for uploading images or videos and viewing the results of the analysis.

## Documentation
Here's the link to the AWS Rekognition documentation:
https://docs.aws.amazon.com/rekognition/index.html

The documentation includes detailed guides, tutorials, API references, and sample code to help you get started with Rekognition. It also covers a wide range of topics, including image and video analysis, data privacy, security, and performance optimization.
AWS for Machine Learning
Tags: aws, css, cloudcomputing, machinelearning
Published: 2023-04-25 05:37:21

AWS offers a variety of services for machine learning, and the best one for you will depend on your specific needs and use case. Here are some popular AWS technologies for machine learning:

1. Amazon SageMaker: SageMaker is a fully managed service that provides developers and data scientists with the ability to build, train, and deploy machine learning models at scale. It includes a number of pre-built algorithms, frameworks, and development tools to help you get started quickly.

2. Amazon Rekognition: Rekognition is a computer vision service that allows you to analyze images and videos to identify objects, people, text, and activities. It can be used for a variety of use cases, including facial recognition, content moderation, and media analysis.

3. Amazon Comprehend: Comprehend is a natural language processing (NLP) service that allows you to analyze text data to identify key phrases, entities, sentiment, and more. It can be used for a variety of use cases, including social media monitoring, customer feedback analysis, and content moderation.

4. Amazon Transcribe: Transcribe is a speech-to-text service that allows you to transcribe audio and video files into text. It can be used for a variety of use cases, including call center analytics, video captioning, and voice-controlled applications.

5. Amazon Translate: Translate is a neural machine translation service that allows you to translate text between languages. It can be used for a variety of use cases, including website localization, customer support, and content creation.

6. Amazon Personalize: Personalize is a service that allows you to create personalized recommendations for your customers based on their behavior and preferences. It can be used for a variety of use cases, including e-commerce product recommendations, content recommendations, and personalized marketing campaigns.

These are just a few examples of the many machine learning services that AWS offers. It's important to evaluate your specific needs and use case to determine which service will work best for you.

More articles are coming in the series. Stay Tuned!
Global Testers Summit 2023
Tags: qatouch, codequality, testing
Published: 2023-04-25 05:08:05

I am delighted to invite all my gracious community members to be a part of the Global Testers Summit 2023 hosted by QA Touch on May 15th and 16th. I look forward to welcoming you all to the gathering and trust me I will be happy to see your faces and I really mean it :) https://lnkd.in/gt2bTxn6 (Block your calendar) #globaltesterssummit #qa #testers #automationtesting #manualtesting
How to Programmatically Get All Websites in Magento 2
Tags: webdev, magento, magento2tutorials
Published: 2023-04-25 03:55:42

Magento 2 is a vigorous and adaptable web based business stage that gives many elements and instruments for engineers to make and oversee online stores. As a designer, you might have to recover a rundown of all sites in Magento 2 for different purposes, like relegating classes, refreshing site settings, or making new stores. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/afl84jb4otswkgpi3y2d.png)

In this blog, we will plunge into the subtleties of how to automatically get all sites in Magento 2. We will utilize the Magento\Store\Model\StoreManagerInterface class, which is an underlying class in Magento 2 that gives admittance to all store-related information and usefulness. This class empowers us to recover a rundown of all sites accessible in the Magento establishment and perform different procedures on them. 

Toward the finish of this blog, you will have a reasonable comprehension of **_how to programmatically get all websites in Magento 2_**. Moreover, you may likewise need to know how to introduce an expansion physically in Magento 2, which is a valuable expertise for designers to have.

For more info, visit **_https://webguru.dev/how-to-programmatically-get-all-websites-in-magento-2/_**
Lubridate Package in R
Tags: datascience, tutorial, beginners, learning
Published: 2023-04-25 03:39:18

In this post, we take a look at the Lubridate Package in R. You use it with the date specific code in the R Language. 

Most of the datasets that deal with the time, you should consider making use of the lubridate to deal with the time and date with those. 

I created the video that covered the features of the lubridate package. You can take a look at the video below.

{% embed https://www.youtube.com/watch?v=qm3PAylzBLE %}

Let's take a look at how it can be installed. 

`install.packages(lubridate)`

and then make sure to use the library in the code. 

`library(lubridate)`

There are many ways we can make use of the lubridate to work with dates. So here is one such that you can make use of as well. 
`
bday <- dmy("22/3/1984")
month(bday)`

and say that you want to work with the year. 

`year(bday)`

There are many other ways to work with it say using timezone, setting your own date, getting system date and so on. It's upto you on how you can choose to get the data. 

I recommend playing with some documentation examples. 

I hope this helps anyone who is trying to learn the data science and exploring the dataset on their own.
UPDATE - Side Project: Content Sharing Platform
Published: 2023-04-25 03:27:32

The web app I am developing allows educators to browse, discover, store and share various educational resources such as articles, documents, presentations, etc. I'm using Java and React while practicing DevOps. The app will help educators facilitate children's learning.

So far, I completed the research and design phase. I started developing the backend API and included the entities you would expect such as EducationalResource, Subject, Topic, Skill, etc. I came up with a [basic prototype](https://login.adambahri.com:1112/realms/jhipster/protocol/openid-connect/auth?response_type=code&client_id=web_app&scope=openid%20profile%20email&redirect_uri=https://app.adambahri.com:1114/login&login_hint=user&password=cca86cd5b97b) and I dockerized the web app. 

I am currently working on the frontend (multipage React app), including a submit form using React Hook Forms and PrimeReact UI component library. As for the backend, I'm progressing with features such as notifications, voting, favorites, and sharing.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0x0h3ngttm8oei7w8ohl.png)

There is still much to be done. I'll continue working on the submit form, search box, search results, side panel (faceted search), navigation and user/moderator dashboard. I'm also planning for a better search experience by adding semantic search and a better author workspace with inbuilt note taking.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/geo0lddor49hx7bu33fs.png)

I aim to enter beta testing phase at the end of the year and I'll create an online community to gather feedback.

Future plans: improve search capabilities by integrating AI language models, end-to-end testing and scalability testing.

As my time is limited, I am open to contributions from individuals with skills in backend development (Java/Spring Boot/Elasticsearch), frontend development (React), or UI/UX design. If you are interested, please don't hesitate to contact me.
What is data lineage and why is it relevant?
Tags: datalineage, data
Published: 2023-04-25 03:19:00

[https://belowthemalt.com/2023/04/24/what-is-data-lineage-and-why-is-it-relevant/]
(https://belowthemalt.com/2023/04/24/what-is-data-lineage-and-why-is-it-relevant/)
10 best practices for ensuring proper monitoring and observability
Tags: devops, devjournal, monitoring, distributedtracing
Published: 2023-04-25 02:11:00

Monitoring and observability are critical for maintaining the performance and reliability of a system. 

Here are 10 best practices for ensuring proper monitoring and observability:



![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pektwvznadrgaszvd0sf.png)


## 1. Define clear performance and availability targets: 
Define specific goals for the performance and availability of your system, and use these targets to measure and evaluate your monitoring and observability efforts.


## 2. Monitor the system end-to-end: 
Monitor the entire system, from the frontend user experience to the backend infrastructure, to ensure that you have a complete view of the system's performance and behavior.


## 3. Use a variety of monitoring tools and techniques: 
Use a range of monitoring tools and techniques, including metrics, logs, and tracing, to gain a more comprehensive understanding of the system.


## 4. Monitor the performance of individual components: 
Monitor the performance of individual components, such as databases, microservices, and APIs, to identify and troubleshoot issues at the source.



## 5. Use alerts and notifications wisely: 
Use alerts and notifications to notify you of potential issues, but avoid overloading yourself with unnecessary alerts. Use alerting thresholds and other techniques to ensure that alerts are actionable and relevant.



## 6. Visualize your data: 
Use dashboards and other visualizations to make your monitoring data more accessible and easier to understand.


## 7. Enable log analysis and searching: 
Use log analysis and searching tools to quickly and easily identify and troubleshoot issues in your logs.


## 8. Implement distributed tracing: 
Use distributed tracing to track the end-to-end flow of requests through a distributed system, providing detailed information about the performance and behavior of the system.



## 9. Monitor your monitoring: 
Monitor the performance and availability of your monitoring and observability tools, to ensure that they are functioning properly and providing accurate and reliable data.



## 10. Continuously improve: 
Regularly review and evaluate your monitoring and observability practices, and make improvements as needed to ensure that they are effective and meet your needs.


By implementing best practices  organizations can effectively monitor and observe their systems and proactively identify and address potential issues before they impact users. 


Thanks for reading this.

If you have an idea and want to build your product around it, schedule a call with [me](https://calendly.com/pragyanatvade).

If you want to learn more about DevOps and Backend space, [follow me](https://dev.to/pragyanatvade).

If you want to connect, reach out to me on [Twitter](https://twitter.com/pragyanatvade) and [LinkedIn](https://www.linkedin.com/in/pragyanatvade/).
Embracing the Peaks and Valleys of Progress.
Tags: webdev, javascript, beginners, programming
Published: 2023-04-25 01:38:46

Day 84 of #100DaysOfCode, and today I continued working on the Contractr project. I started off by writing the code to ensure that matches don't show up in the cards to swipe on, which worked well. I then wrote the logic for the heart button to trigger the swipe function, which unfortunately introduced a bug. I spent some time debugging it, but eventually realized that it was time to call it a night and get some rest.

It can be frustrating when you encounter unexpected bugs and spend hours trying to solve them, but I've learned that it's all part of the process of coding. There are highs and lows in any project, and it's important to be honest about both.

Despite the challenges, I'm still making progress on the Contractr project, and I'm excited about the possibilities it holds. I'm learning new things every day and building my skills as a developer. I'm grateful for the support and encouragement I've received from the coding community, both online and offline.

Tomorrow is a new day, and I'll show up to continue the work. It's all part of the journey, and I'm ready to embrace it, bugs and all.
Tutorial - How to present Google slides on a Google Meet call
Tags: slides, googlemeet, googleslides
Published: 2023-04-25 01:10:41

#google/meet #tutorial #presentations 

## Notes
Slideshows expand to take up full screen when in presentation mode. To be able to 

## Steps

1 - Need 2 monitors
	one for the slideshow 
	one  for the call -  so you can see what others are seeing
2 - Use chrome browser (will not work with other browsers)
	Open 2 chrome windows
		one for slideshow
		one for call 
3 - Place all content you want to share in one screen
	i.e. 
		slideshow window 
		code IDE
		Simulator device 
		etc
4 - In second screen place chrome browser window with call 
5 - Join meeting 
6 - on meeting window, click on present, and then on A TAB

7 - Select the Google slides tab, click on SHARE 

8 - In Google slides tab , click on SLIDESHOW to start presenting 
	control slideshow presentation from the slideshow window
	Tip: To switch to sharing other screens, chose those screens specifically, not entire windows
Survey: Transformer-Based Named Entity Recognition on Drone Flight Logs to Support Forensic Investigation
Tags: nlp, deeplearning
Published: 2023-04-25 01:09:45

## 選定理由

ドローンフォレンジックという新しいドメインだが、法規制の解除によって社会需要は増加すると予想。対象タスクは自然言語処理の固有表現抽出。

Paper: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10007817
Code: https://github.com/swardiantara/droner-cosine

## 概要

[社会課題]ドローンの増加に伴い、ドローンの事件や攻撃の発生数が増えていることから、事故の防止、原因の調査・究明の機会が増加している。事故発生時にはフォレンジック調査が行われ、事件のシナリオを明らかにし、リスクを軽減し、調査結果の報告が必要となる。
[技術課題]ログメッセージとタイムスタンプから構成される法科学的タイムラインを構築するためにフライトログデータシーケンスの文脈依存性を捉える必要がある。
[提案]自己注意機構にてドット積の代わりにコサイン類似度を使用したTransformerエンコーダを提案し、これに対しCNN文字埋め込み、BERT単語埋め込み、Transformerエンコーダ、およびCRFデコーダを持つ新しいNERアーキテクチャを提案した。又、ドローンのフォレンジックというドメインにおいて、6つのエンティティタイプを定義した新しいNERデータセットを構築した。
[効果]過去のベースラインモデルを超え、F1スコアが91.348％となった。

## 提案内容

ドローンのフォレンジック分野において、新しいNERデータセットを構築


## 実験結果と分析

前処理として収集されたログメッセージをトークン化して、大文字小文字を区別するためにオリジナルのメッセージを保持し、ドットとカンマを文脈の区切り役として保持しながら、CoNLL形式に変換することで、標準的なNERデータセット形式に変換し、等しい長さのトークンとラベルを埋め込み層に入力して表現特徴ベクトルを取得する。

![fig10](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pwx1s68jq3s2gv08cuaa.png)

図１０は他ベースラインモデル比較である。比較対象はCNN-BERT-Scaled Transformer、AdaTransBERT-Unscaled Cosine、AdaTrans-BERT-Scaled AdaTrans、AdaTrans-BERT-BiLSTMのエンコーダーモデルで、すべてについて、デコーダーとしてCRFが使用されている。収束速度に関して、提案手法（Cosine）はTransformerおよびAdaTransと同様に速く収束した。図10を見るとRecallに関して提案手法(Cosine)はTransformerを超えているが、F1では劣っている。一方で、未スケーリングのAdaTrans attentionは、AdaTransを文字埋め込みとして、BERTを単語埋め込みとして使用し、90.514％のF1スコアで第三位に位置している。これは、データセットが平均的にトレーニングデータとテストデータの両方で、6.3語と8.8語の長さしかないため相対アテンションメカニズムが機能しにくいためであると考えられる。

![fig1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x35s5zcxccudguwt1vc8.png)

上記は固有表現抽出をしたサンプルである。Component, Action, Parameter,
Function, State, そして Issueの６種類のエンティティである。

課題としてVTO Labs Drone Forensic データセット以外に、15以上の異なるモデルと20以上のデータセットからも、2000未満のログメッセージしか見つからず、さらに、データセットには正常パターンのみで、フォレンジック調査、発見、および報告の観点から提案手法をテストするための異常パターンが存在しない点が挙げられる。
Specialization vs. Generalization: Which Is Better for Programmers?
Tags: discuss, beginners, codenewbie
Published: 2023-04-25 00:33:37

Are you a programming language specialist or a generalist? Do you prefer to focus on one language and become an expert or learn multiple languages and be a generalist?

As you develop your coding skills, you'll face the decision of whether to specialize in one programming language or learn multiple languages. Which will you choose? A depth of knowledge? Or versatility? 

Share your thoughts on this debate in the comments below!

---

Follow the [CodeNewbie Org](https://dev.to/codenewbieteam) and [#codenewbie](https://dev.to/t/codenewbie) for more discussions and online camaraderie!
Your Cover Letter - Your perfect cover letter using the AI
Tags: saas, hr
Published: 2023-04-25 00:31:31

49% of recruiters want job seekers to submit their cover letters, but writing them should not feel like homework. Use our AI cover letter generator to get your free, perfect cover letter in seconds, tailored to the job position and the company you're applying to! It also generates a final PDF you can download and send to the hiring manager. Like magic! Isn't it?

[https://yourcoverletter.com](https://yourcoverletter.com)
As quatro fases do Splunk: input, parsing, indexing e searching.
Tags: logs, splunk
Published: 2023-04-24 23:53:56

Voltando aqui para avançar um pouco mais nos conceitos do Splunk. No post anterior, mostrei como fazer o deploy do Splunk no seu computador pessoal. Você pode conferir no link [aqui](https://dev.to/lanussemorais/meu-primeiro-lab-com-splunk-a1f).

---

Já abordei o que é o Splunk no primeiro post, então, nesse post, vou partir do princípio que você já ouviu falar da ferramenta.

Do ponto de vista macro, o Splunk tem quatro fases para o tratamento do dado: input, parsing, indexing e searching.

A primeira fase é o input, onde os dados são coletados de diferentes fontes, como arquivos, serviços, agentes e outros. Nessa fase, estamos preocupados apenas em enviar os dados sem muita formatação. O importante é garantir que cheguem.

Depois que os dados são coletados, o Splunk passa para a fase de parsing, onde os dados são processados para extrair informações importantes. Também nessa fase, os dados são convertidos em eventos, e você pode personalizar as configurações para atender às suas necessidades específicas, talvez mudar o nome do host. Sua imaginação é o limite.

A próxima fase é a fase de indexing, onde os eventos gerados na fase anterior são indexados, criando índices que apontam para a localização dos dados no disco. Isso permite que você pesquise e recupere os dados de forma rápida. O Splunk indexa os eventos em tempo real, o que significa que os eventos são disponibilizados para pesquisa e análise assim que são coletados.

Por último, mas não menos importante, temos a fase de searching, que possibilita que você pesquise e analise os dados indexados. O Splunk fornece uma interface web para pesquisa de dados (aquela interface do primeiro post), permitindo que você pesquise dados usando palavras-chave, expressões regulares e outros recursos avançados de pesquisa oferecidos pelo SPL. Você pode visualizar os resultados da pesquisa em tabelas, gráficos e outros formatos para entender e analisar melhor seus dados. Com a opção de pesquisa em tempo real, você pode ver os resultados da pesquisa à medida que os dados são coletados.

Em resumo, com uma visão de alto nível, é assim que seu dado sai do seu servidor e vai parar no Splunk como um evento.

Caso você queira mais informações sobre as fases do processo, pode conferir a documentação oficial do Splunk: 
https://docs.splunk.com/Documentation/Splunk/9.0.4/Deploy/Datapipeline