Understand how a DB work? - Page 1
Forum section: electronics / general

author: Lord of nothing
author_rank: Super Contributor
timestamp: December 07, 2020, 12:31:58 am
content: HiSorry for that stupid question but how does a Database work internally?Does the is look like a CSV File? Does the Data get save in some random order?Sry I try to understand what make a DB different to a CSV File and or other Files who contain similar information.Thanks

author: ataradov
author_rank: Super Contributor
timestamp: December 07, 2020, 01:02:51 am
content: There are entire books dedicated to the subject.On a large scale, what makes them different is efficient indexing of the data. Plus the actual format is obviously binary, not the text-based as CSV. And the data is arranged for effective access (cache friendliness).And the CSV file is just that, a file. Same as a database file. The difference here is that database file is meant to be read by the corresponding database software. Unlike with CSV, there is no universal database format, each software decided what format it wants.And the database does not just contain the data itself, it can also contain other meta-information about data types, stored procedures, triggers.

author: WattsThat
author_rank: Frequent Contributor
timestamp: December 07, 2020, 01:51:38 am
content: One of the simplest to understand databases is Btrieve. It’s just a record manager, nothing more.There is some info about it on Wikipedia.https://en.m.wikipedia.org/wiki/Btrieve

author: Nominal Animal
author_rank: Super Contributor
timestamp: December 07, 2020, 02:16:02 am
content: If I understood correctly, Lord of Nothing is also learning Python?Python 3 has a built-insqlite3module.sqliteitself is a very small SQL but pretty darn powerful and full-featured database engine written in C as a library. There is no separate server at all: the library implements the entire database engine. Usually, you use a file (sqlite's own format) to store the database.There are lots of examples on Python sqlite3 on the net. The most important thing, I believe, is to make sure you executeparametrizedstatements instead of literals. That is, although SQL is text, you don't build text strings containing the data or parameters, you use the placeholder mechanism. There are two methods for this in Python sqlite3. One is to use a single question mark for each component in order, and pass a list, tuple, or iterable to the execute statement after the string. The other is to use:namein the SQL statement instead of the string, and then pass a dictionary, e.g.{ "name":"value", "othername":"othervalue" }.The reason for this is that if you just build SQL strings with the values in them, the values may contain extra characters that are interpreted as SQL statements instead of as a value. Theoretically, you can use a helper function thatescapesall those, but it is always risky. The placeholder mechanism is superior.Obligatory XKCD. (If using the placeholder mechanism, there would be no problem with Little Bobby Tables. But with the string-building method, if your escaping isn't perfectly on point, instead of querying or modifying the record for Little Bobby Tables, you lose the table named Students, if the database has one, and there is no privilege separation to stop it [and usually there is not].)I would suggest that you do a web search onPython sqlite3, and pick one that fits your needs best; but first, skip to the part where they do.execute(...)and make sure they use the two-parameter form (that's the placeholder mechanism), instead of building full SQL statements as plain strings.A quick look at the first couple of matches indicatessqlitetutorial.net/sqlite-pythondoes it right, but the tutorialspoint.com/sqlite/sqlite_python.htm and pynative.com/python-sqlite/ do it wrong; although thepynative.com/python-sqlite-blob-insert-and-retrieve-digital-data/subpage of the latter does it right again.I would recommend doing some of the tutorials and experiments (say, a parts catalog) first, and only then looking at theSQLlanguage itself – the underpinnings of SQL-based databases like SQLite and PostgreSQL. That includes things like learning how to design effective and fast databases, efficient complex queries, and so on.

author: bobcat2000
author_rank: Regular Contributor
timestamp: December 07, 2020, 11:11:39 pm
content: You will need to go to college to learn it. I don't think they teach this in a trade school. Trade school only teach you how to use it.This is the book they use to teach database systems in my school.https://www.amazon.com/Introduction-Database-Systems-8th/dp/0321197844Sorry, I am not too helpful here.

author: bobcat2000
author_rank: Regular Contributor
timestamp: December 08, 2020, 01:00:08 am
content: This has been so long ago and far far away.According to my book, the records are stored in the file system as pages. The database system requests the file system to locate the page with the records. The file system then requests the disk system to locate the file or files that has the page(s). The disk system return back to the file system and back to the database system.This is up to you to design how you want to store the records in the file system.