teleren/devto huggingface dataset:
Kafka Connect sink to OpenSearch/ElasticSearch: how to sink unix timestamps
Tags: apachekafka, kafkaconnect, opensearch, elasticsearch
Published: 2023-09-13 11:00:00

---
title: Kafka Connect sink to OpenSearch/ElasticSearch: how to sink unix timestamps
published: true
date: 2023-09-13 11:00 UTC
tags: apachekafka, KafkaConnect, OpenSearch, ElasticSearch
canonical_url: https://ftisiot.net/posts/kafka_connect_sink_elasticsearch_opensearch_timestamp/
---

When sinking unix timestamps from Apache Kafka to OpenSearch/ElasticSearch using the dedicated connector, they are not recognized by default as timestamp in the target tech. 

Using the [TimestampConverter SMT](https://rmoff.net/2020/12/17/twelve-days-of-smt-day-8-timestampconverter/) you can change the format into one that is recognized natively by OpenSearch/ElasticSearch.



## Use the TimestampConverter SMT to translate unix timestamps into strings

To have OpenSearch/ElasticSearch automatically recognizing a timestamp we need to push it in the [ISO 8601 format](https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations) `yyyy-MM-ddTHH:mm:ss.SSSZ` where:

* `yyyy-MM-dd` are the year, month, day
* `T` is the string `T`
* `HH:mm:ss.SSS` are the hour, minutes, seconds, milliseconds
* `Z` is the string `Z`

You can achieve this by adding a TimestampConverter SMT to the Kafka connect sink definition with:

```json
"transforms.transform-name.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value"
"transforms.transform-name.field": "ts_ms",
"transforms.transform-name.format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"transforms.transform-name.target.type": "string"    
```

Where:

* `transform-name` is the transformation name, the same name must be referenced in the `transforms` parameter
* `"transforms.transform-name.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value"` will extract the timestamp from part of the message value
* `"transforms.transform-name.field": "ts_ms"` identifies the field `ts_ms` as the one to be converted from unix timestamp
* `"transforms.transform-name.format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"` defines the target ISO 8601 format
* `"transforms.transform-name.target.type": "string"` defines the target field as string (OS/ES will automatically recognize it as timestamp)
Hackoberfest is coming! Are you going to participate?
Tags: opensource, hacktoberfest, hacktoberfest23
Published: 2023-09-13 10:38:15

Hacktoberfest is just around the corner, and I'm thrilled! 🎉 Are you gearing up to participate in this fantastic open-source extravaganza? 🤩 It's a whole month dedicated to contributions, collaboration, and coding brilliance. 💻🌐

Whether you're a seasoned developer or a coding newbie, Hacktoberfest welcomes everyone. It's not just about the pull requests; it's about building connections, learning, and making a positive impact on the open-source community. 🤝✨

 Check out this comprehensive article on HACKTOBER-FEST: https://lnkd.in/dgjGfqpD which covers everything you need to know!

Drop a comment below and let me know if you're in! 🙌👇


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xyc48xy8qi99izeyqvxh.png)
iOS Tips Xamarin Forms - Get Safe Area Height
Tags: xamarin, ios, dotnetmaui, mobile
Published: 2023-09-13 10:36:37

I needed to get the Safe Area height so I could manipulate a screenshot taken on a device. [Stack Overflow](https://stackoverflow.com/a/58856891) to the rescue once again.

 *Yes the project I'm working on has not started the migration to Maui.*

## Xamarin Forms

Setup the interface.

**IGetSafeArea**

```csharp
public interface IGetSafeArea
{
    double GetSafeAreaTop();
    double GetSafeAreaBottom();
}
```

## Xamarin.iOS

Implement the interface and register it.

**GetSafeArea.cs**

```csharp
public class GetSafeArea : IGetSafeArea
    {
        public double GetSafeAreaBottom()
        {
            if (UIDevice.CurrentDevice.CheckSystemVersion(11, 0))
            {
                UIWindow window = UIApplication.SharedApplication.Delegate.GetWindow();
                var bottomPadding = window.SafeAreaInsets.Bottom;
                return bottomPadding;
            }
            return 0;
        }

        public double GetSafeAreaTop()
        {
            if (UIDevice.CurrentDevice.CheckSystemVersion(11, 0))
            {
                UIWindow window = UIApplication.SharedApplication.Delegate.GetWindow();
                var topPadding = window.SafeAreaInsets.Top;
                return topPadding;
            }
            return 0;
        }
    }
```

**AppDelegate.cs**

```csharp
 public partial class AppDelegate : FormsApplicationDelegate
    {
       ...
       DependencyService.Register<GetSafeArea>();
       ...
    }
```

## Using It

```csharp
 if (Device.RuntimePlatform == Device.iOS)
       {
           var safeAreaTop = DependencyService.Get<IGetSafeArea>().GetSafeAreaTop();
          
       }
```

## References

[StackOverFlow](https://stackoverflow.com/a/58856891)
WISECP iRed Mail Module
Tags: webdev
Published: 2023-09-13 10:20:35

WGS iRedmail WISECP Module is a comprehensive mail server solution compatible with both the paid and free versions of iRedmail. It is developed to streamline the process of offering email hosting to your clients by providing automated provisioning of email accounts. With the [WGS iRedMail Module](https://whmcsglobalservices.com/iredmail-for-wisecp/), you can quickly add domains and create, suspend, and terminate email accounts, making your email hosting services more professional and efficient.
Support PIN Server Module
Tags: webdev
Published: 2023-09-13 10:19:16

The [WISECP Support PIN Server](https://whmcsglobalservices.com/support-pin/) Module is designed to enhance customer support processes by implementing a secure and efficient method for verifying customer identity and account access. Consequently, support agents can now conveniently inquire about the customer’s Support PIN, streamlining the verification process while maintaining a professional approach.
Latest SBCL (Common Lisp implementation) improvements in 2022-2023: parallel GC, shared library and more
Tags: lisp, programming
Published: 2023-09-13 10:16:38

The latest big news is that **SBCL got a parallel garbage collector** (available under a build flag). That's awesome news, also something that *people on the internet* were complaining about the lack of (although [you could find one](https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-common-lisp-and-parallel-gc/) in Allegro CL and Clasp (newest, long to build) as well as ABCL with a suitable JVM (with ZGC or Shenandoah)).

- discussion: https://news.ycombinator.com/item?id=37296153

Other great achievements from last year:

- SBCL is callable as a shared library (see [sbcl-librarian](https://github.com/quil-lang/sbcl-librarian/))

- [sb-simd](https://github.com/marcoheisig/sb-simd): [example](https://gist.github.com/digikar99/13ed3868993ca8c71c91b50c8022a64d), 

- [prebuilt binaries for SBCL Common Lisp on Android](https://github.com/bohonghuang/sbcl-termux-build) (termux, unofficial)

- better image compression using zstd

- I'll add https://github.com/sionescu/sbcl-goodies, binaries with "goodies" inside (OpenSSL, libfixposix)

yay!


bonus from 2021: true static binaries are coming: https://www.timmons.dev/posts/static-executables-with-sbcl-v2.html Currently, you can build a binary, but unlike Rust or Go you need a compatible Glibc on the target OS. If you build on a relatively old Linux it works on the latest ones.
HP Elitebook 1030 G3
Tags: oldcomputers, oldlaptops, usedcomputers, refurbishedcomputers
Published: 2023-09-13 10:15:14

**Brand:** HP
**Model:** Elitebook 1030 G3
**Ram:** 8GB
**Hard disk:** 512 GB, SSD
**Processor:** core i7 8th GEN.
**Screen:** Touch screen
**Warranty:** 2 Months
**Price:** Rs. 29,999/-
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gn3vuhpuz56udtcxllt5.jpg)

For more details, please visit our website: [Web IT World](https://webitworld.in/)
Use Bard API (Palm API) without an API key
Tags: ai, productivity, chatgpt, bard
Published: 2023-09-13 10:15:02

Google's Bard is eagerly expected to join the LLM API race so we can all use it in our AI related products. Problem is, currently the access to Palm API Beta is severely limited. Early US based applicants got the key. For non-US applicants and now also for anybody from the US, it's next to impossible to get the key. 

Many of us don't have the opportunity to explore how Bard and Palm API functions. Even if you have Bard web access - API access and regular web based use differ so you really need to query the Palm API to experience what it provides. Also, web based use isn't available for everyone. This is where APIScout.AI comes to play. 

[APIScout.AI compares ChatGPT API and Palm API output](https://apiscout.ai). It is displaying response to your prompts side by side, querying simultaneously the two biggest LLM APIs out there. It allows full API request configuration, and is free, can be used without registration. This way you can check out Bard without having a key or web access. 

There's a limit to the number of prompts you can enter though. But it's enough to understand what Bard API offers so you can plan ahead to use it when it becomes commercially available. When you run out of prompts you can enter, you can continue comparing ChatGPT and Bard if you have your own API keys to enter into the APIScout.AI's GUI. 

Please let us know in the comments if you find any other way to access Palm API at this time, or if Beta registrations continue.
Django's Object-Relational Mapping (ORM)
Tags: django, orm, python, webdev
Published: 2023-09-13 10:11:24


**Django's Object-Relational Mapping (ORM)** is a powerful feature that simplifies database interactions in Django applications. It allows developers to work with databases using Python classes and objects rather than writing SQL queries directly. Here's a complete overview of Django's ORM:

**1. Model Definition:**

The foundation of the Django ORM is the definition of data models. Models are Python classes that represent database tables. Each model class maps to a table, and its attributes define the table's columns.

```
from django.db import models

class MyModel(models.Model):
    name = models.CharField(max_length=100)
    age = models.IntegerField()

```


**2. Fields:**

Django provides various field types (e.g., CharField, IntegerField, DateField) that correspond to SQL data types. You can use these fields to define the structure of your database tables.


**3. Model Relationships:**

Models can define relationships with other models. The most common types of relationships are:

ForeignKey: Represents a one-to-many relationship.
OneToOneField: Represents a one-to-one relationship.
ManyToManyField: Represents a many-to-many relationship.

```
class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

```

**4.Database Table Creation:**

Django automatically generates database tables based on your model
 definitions. You can create these tables by running migrations:

```
python manage.py makemigrations
python manage.py migrate

```

 **5.Querying the Database:**

Django's ORM provides a QuerySet API for querying the database. You can filter, order, and aggregate data using a Pythonic syntax.

```
# Create a new object
new_obj = MyModel(name='John', age=25)
new_obj.save()

# Retrieve an object
obj = MyModel.objects.get(id=1)

# Update an object
obj.age = 26
obj.save()

# Delete an object
obj.delete()

```
**6. CRUD Operations:**

You can easily create, retrieve, update, and delete records using the model's methods:

```
# Create a new object
new_obj = MyModel(name='John', age=25)
new_obj.save()

# Retrieve an object
obj = MyModel.objects.get(id=1)

# Update an object
obj.age = 26
obj.save()

# Delete an object
obj.delete()

```
Django's ORM offers advanced features like migrations, database indexes, database routing, database migrations, and more to handle complex scenarios in real-world applications.

In summary, Django's ORM simplifies database interactions in Django applications by allowing you to define models, perform database operations, and write queries using Python code, making it easier to build robust and maintainable web applications
Letter to Prof. Turtle 1: Intro to Coding
Tags: coding, softwaredevelopment, buildinpublic, learning
Published: 2023-09-13 10:10:58

> **Welcome to 'Letter to Prof. Turtle' Series!**

> Hey there, Dev.to community! ✨

> This is a new series of articles, in collaboration with [Prof. Turtle,](https://dev.to/seif_sekalala_81e09fe6b9e) to help you dive into the world of coding and software development. 💻

> Stay tuned for more insightful discussions aimed at getting you started on your coding journey! 📚💡


Dear [Prof. Snr-Turtle,](https://dev.to/seif_sekalala_81e09fe6b9e)
  
I hope this message finds you well. I'm excited to begin this series of conversations where I, Mr. Jnr.-Hare, will be seeking your guidance on various coding-related topics. 

To kick things off, I'd like to delve into the fundamental question: _"Why and How to Get Started with Programming?"_

Let's start at the beginning. Why is programming important, and how can I, as a beginner, embark on this coding journey?
 

## Why Learn Programming

Programming, [Prof. Snr-Turtle](https://www.linkedin.com/in/seif-sekalala-430b8a21/), seems like a complex world. Could you help me understand why it's worth diving into? What doors can it open for me?
 

## How to Begin
Assuming I'm completely new to this, what's the first step? How do I even start learning to code? I need a simple roadmap.
 

## Challenges I Might Face

I've heard coding can be challenging. What kind of obstacles should I be prepared for, and how can I overcome them?
 

## Realistic Expectations

Can you tell me how long it might take before I can do something useful with code? I want to set realistic expectations. (SMART Goals)
 

## Choosing the Right Language

With so many programming languages out there, how do I choose the right one for a beginner like me? What's the language to start with?
 

## Learning Resources

Do you have any recommendations for beginner-friendly learning resources or websites where I can get started?
 
Before we wrap up, please summarise the key points for me. What should I remember as I begin this coding journey?
 

## Next Steps

Thank you for sharing your insights, [Prof. Snr-Turtle](https://www.linkedin.com/in/seif-sekalala-430b8a21/).

Looking forward to our following conversation.
 
_Best regards,
Mr. Jnr.-Hare._
Unreachable static website distributed via AWS CloudFront
Tags: aws, s3, cloudfront, website
Published: 2023-09-13 10:07:11

**DISCLAMER**: The following was a very specific situation I find myself in when my own website hosted in an AWS S3 bucket and distributed via AWS CloudFront, stopped working. Here is how I solved the issue smoothly.

---

I own a very simple personal website and I used to host it in an AWS bucket.

👉 This is it, [Minimal Gap](https://cloud.minimalgap.com/)!

Distributing using my own domain just required to enable `Static website hosting` in AWS S3 Bucket properties and play a little bit with dns.

Here is a quick summary of how it was done.

* S3 Bucket Name: minimalgap.com
* DNS setup: cloud 	 CNAME	minimalgap.com.s3-website-eu-west-1.amazonaws.com

As a further step when I put it online, I decided to distribute it using a CDN, of course the easiest step was to leverage AWS CloudFront.

To do that I created an AWS distribution, I left most of the options as they are, just changed the following properties:
* `Origin Settings`, for `Origin Domain Name`, enter the bucket website endpoint. You get this from the `Static website hosting` section of `Properties` for the Amazon S3 bucket. This will result in `Origin Type` being `S3 static website`.
* I optionally generated a TLS certificate via AWS, to hold a certificate related to my own domain (minimalgap.com).

All of the above is well documented [here](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/getting-started-cloudfront-overview.html#getting-started-cloudfront-distribution-root)

However, out of nowhere, my CloudFront Distribution stopped working. After a lot of digging I found out that some AWS configurations may have changed internally and my distribution was lacking of  “something”, that prevent it to work correctly.

It was an issue with the Origin configuration itself. Indeed:
* picking my own distribution in AWS Cloud Front and clicking on its ID,
* getting to the view `Origins`, I found the correct configuration pointing to my S3 bucket,
* trying to select and edit the problem was clear: the protocol selection was missing (HTTP, HTTPS, Match viewer).

Once I selected one of the values the distribution was back working as expected without any issue (see image below).
Therefore that `something` was actually the protocol selection within the Origin configuration!

![Origin settings](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3kc6bp82sht2j15zape.png)

As I said my idea is that AWS changed a little bit some configuration so distributions which were already setup, need to be manually fixed. 
Maybe this will result useful for someone else encountering the same issue.
CoachBase: Discover the Right Online Coach for Personal Growth"
Published: 2023-09-13 10:06:18

"Welcome to CoachBase, your go-to platform to [find an online coach](https://www.coachbase.io/find-coach) for your personal growth journey. We offer an extensive directory of skilled coaches covering diverse areas like life coaching, career advancement, well-being, and more. Empower yourself by connecting with an expert coach who aligns with your specific needs and goals. With CoachBase, you can explore detailed coach profiles and authentic reviews and make an informed choice to kickstart your transformation. Whether you seek guidance to overcome challenges, unleash your potential, or achieve new heights, our online coaching services cater to all your aspirations. Experience the life-changing impact of personalized coaching with CoachBase—take the first step towards a brighter future now!"
Home Sales Unleashed: Mastering the Self-Listing Revolution
Tags: sellmyhousewithoutarealtor
Published: 2023-09-13 09:59:28

Considering the "For Sale By Owner'' approach to real estate? It's entirely possible to [**sell your house without a realtor**](https://ownersre.com/our_services) and without stress. By leveraging a "sell-your-home-yourself" platform, you can access a plethora of professional features. As offerings differ among providers, it's crucial to research and compare. Choose a service that aligns with your preferences and delivers the support you seek during your sales journey.
Moreover, these service provider platforms emphasise the confidentiality of your data. Rest assured, they do not sell your information to third parties. While you might occasionally receive special offers from partner companies, safeguarding your personal details remains a top priority.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vo3wwahatriy8t3bv7xo.jpg)
Wednesday Links - Edition 2023-09-13
Tags: java, jvm, performance, concurrency
Published: 2023-09-13 09:49:00

Call for Discussion: New Project: Babylon (2 min)🗼
https://mail.openjdk.org/pipermail/discuss/2023-September/006226.html

Problem Details for HTTP APIs - RFC 7807 is dead, long live RFC 9457 (9 min)🚩
https://blog.frankel.ch/problem-details-http-apis/

Spring AI Provides Integration with OpenAI and Azure OpenAI (4 min)🦾
https://www.infoq.com/news/2023/08/spring-ai/

Best distributed task scheduling framework — Openjob 1.0.7 released (4 min)⏰
https://habr.com/en/articles/760108/

Good performance is not just big O (6 min)🏃
https://jmmv.dev/2023/09/performance-is-not-big-o.html

Structured concurrency and scoped values in Java (4 min)🔀
https://softwaremill.com/structured-concurrency-and-scoped-values-in-java/

Bad software examples - how much can poor code hurt you? (6 min)🪅
https://softwaremill.com/bad-software-examples-how-much-can-poor-code-hurt-you/

A systematic approach to debugging (7 min)🕵️
https://ntietz.com/blog/how-i-debug-2023/
Rack Manager WHMCS Module
Tags: webdev
Published: 2023-09-13 09:42:36

WGS introduces a powerful new data center infrastructure management module that has the capabilities to revolutionize the way IT departments manage their data centers.[Rack Manager WHMCS Module](https://whmcsglobalservices.com/whmcs-rack-manager-addon/) With powerful rack manager features, IT staff now have access to intelligent data center information that accurately reflects their IT environment.

All Datacenter inventory like physical, virtual, blade, clustered, and more device types.
What are the super powers of programmers?
Tags: discuss, programming, beginners
Published: 2023-09-13 09:39:10

---
title: What are the super powers of programmers?
published: true
description: Super-powers allow you to make things much better and much faster than what people without those powers can do.
tags: discuss, programming, beginners
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-09-13 09:34 +0000
---

The other day I was watching a series with some people with super-power.

Super-powers allow you to make things much better and much faster than what people without those powers could do.

So I was wondering what super-powers do programmers have? I found 3:

* Deep knowledge of a version control system. (Today it is git).
* The knowledge and the will to write tests.
* The knowledge how to set up a CI system to run the tests on every push and on every pull-request.

What do you think are the super-powers that programmers can have?
Trickle: Transform screenshots into searchable treasures with AI
Tags: ai
Published: 2023-09-13 09:35:02

Trickle is for people who frequently capture screenshots for later use and struggle with managing and finding them from the depths of buried folders.

With Trickle, you can intuitively manage your screenshots as they pile up. You can also extract insights from these images using the reasoning capabilities of GPT-4, turning your screenshots into “visual memos” that are both readable and searchable.


**How Trickle Works**

[1] Uploading Screenshots to Trickle

Simply send your screenshots to Trickle. This helps you declutter your camera roll and prevents your screenshots from accumulating. Give a home to your screenshots, offloading them to one place, and never forget.

[2] AI-Generated Insights

Receive an insightful digest generated by our AI for each uploaded screenshot. Trickle is especially adept at processing text-heavy screenshots, including those unstructured diagrams.

[3] Easy Retrieval and Interaction

Easily browse, search, and query to retrieve your saved screenshots. You can not only instruct the AI to find specific screenshots but also request it to provide additional information or insights.


**🎉 Special Offer for Your 🎉**

Start your $10 plan for just $1 in the first month with our limited-time offer. Visit [trickle.so](https://www.trickle.so) now to get your discount code today!


By the way, we are live on Product Hunt.

Check here: [Trickle on Product Hunt](https://www.producthunt.com/posts/trickle-4)