[solved] Smart Searching and Replacing Text - Page 1
Forum section: electronics / general

author: DiTBho
author_rank: Super Contributor
timestamp: October 29, 2021, 08:41:21 am
content: imagine you have a source code written by someone years ago and the names of variables are just a lettere.g.Code:uint32_t a;uint32_t b;uint32_t c;uint32_t d;...uint32_t m;uint32_t s;uint32_t t;...uint32_t z;Worse still, the source code is something like 25281 lines of C code with thousands of points where a single letter variable { 'a' .... 'z' } is used, and you are asked to restyle the code with better variable names (plus other code-refactoring, etc etc)Silly question, but ... how can you search and replace these variable names without doing it manually?( I am lost )

author: DiTBho
author_rank: Super Contributor
timestamp: October 29, 2021, 08:46:29 am
content: Code:...a = f(a, b, c, d, m, n, 5, 0xf562);d = f(d, a, b, c, m, k, 9, 0xc0b0);c = f(c, d, a, b, m, 14, 0, 0x2061);b = f(b, c, d, a, 0, 20, 0, 0xe9ba);a = f(a, b, c, d, 5, 5, 1, 0xd62d);d = f(d, a, b, c, 10, 9, k, 0x0253);c = f(c, d, a, b, 15, 14, n, 0xd8a1);...a single letter also used for function names, f(...)

author: Ed.Kloonk
author_rank: Super Contributor
timestamp: October 29, 2021, 09:03:08 am
content: Geez.

author: IanB
author_rank: Super Contributor
timestamp: October 29, 2021, 09:06:35 am
content: Use a smart code editor, like for example Visual Studio. It will have built-in refactoring tools to allow variable and function renaming, amongst other things.

author: DiTBho
author_rank: Super Contributor
timestamp: October 29, 2021, 09:17:59 am
content: Quote from: IanB on October 29, 2021, 09:06:35 amI usually write code with { Nano, Vim, Geany }. Is there something similar to Visual Studio but open-source?( not because I hate Microsoft, but because I don't have x86 hardware on hands at the moment, and here I can only run Linux/PPC64)

author: RoGeorge
author_rank: Super Contributor
timestamp: October 29, 2021, 09:18:24 am
content: Find and replace is not the best tool to rename variables. Try another editor that has rename for variables.Most of the modern IDEs/editors should have some sort of "Rename Variable" on the right click, or somewhere in the menus, or in the refactoring area.

author: DiTBho
author_rank: Super Contributor
timestamp: October 29, 2021, 09:35:26 am
content: Quote from: RoGeorge on October 29, 2021, 09:18:24 amYup, I have just read the comment on the MicrosoftQuoteI have to find *which* open-source code-editors have this feature

author: RoGeorge
author_rank: Super Contributor
timestamp: October 29, 2021, 09:44:48 am
content: Quote from: DiTBho on October 29, 2021, 09:17:59 amVisual Studio is open source for many years now.It is called 'Visual Studio Code', it's free and open source.https://linuxize.com/post/how-to-install-visual-studio-code-on-ubuntu-20-04/I'm not a Microsoft fan, but VS Code is one of the best IDEs ever, IMHO, and it has countless plugins and support for any language. At some point I was using it instead of Arduino IDE, with programming and serial console, too, from inside VS code. It was fun to adjust the sources for both the microcontroller part (C++ arduino) and the PC data receiving part (Python) from inside the same IDE.

author: blacksheeplogic
author_rank: Frequent Contributor
timestamp: October 29, 2021, 09:45:02 am
content: Quote from: DiTBho on October 29, 2021, 09:35:26 amI've not used this but try looking up renamec.vim. I think you need to have built cscope database. YMMV.

author: Nominal Animal
author_rank: Super Contributor
timestamp: October 29, 2021, 10:31:57 pm
content: I think Visual Studio Code is the easiest option, and makes most sense here.You could also usecscope(either from package management, orupstream). It provides a vim-like editor, and lets you locate and edit each instance; but AFAIK does not provide a batch replacement interface.If you create filecscope.filescontaining the (relative) paths to the C source and header files affected, and runcscope -bc, cscope generates an ASCII line-based databasecscope.out, that a simple UI could use to make the changes in parallel, showing each change (with a few lines of context) to the user to verify whether the replacement should be applied or not, advancing through the source files in line order; only asking the user whether each change should be applied (y/n/q, for Yes, No, or Quit). (I would "store" the changes, and not apply them directly, so that the user could also go back to earlier decisions and change them, and only apply the changes after all choices (to all, or just the current file) have been made. This reduces the risk of bugs due to editing the file and making the database out of date.)

author: DiTBho
author_rank: Super Contributor
timestamp: October 29, 2021, 11:00:12 pm
content: I solved with a library that I developed years ago to tokenize a sourceIt understands C-delimiters and line-numbers, so today I wrote an application that accepts "apply from this line of code to this line", compares each token with with a given pattern, and in case of match it renames the token' name but not its properties.- read the code block range- tokenize- within the block, compare token' name and replace- reassemble and print outI spent 2 hours on it, it worked insanely great at the first tryI have also looked atthislist of IDEs, not yet complied, but CodeLite looks interesting.Quote

author: Ed.Kloonk
author_rank: Super Contributor
timestamp: October 29, 2021, 11:44:29 pm
content: Wow.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 30, 2021, 04:44:10 am
content: I use Code::Blocks, which is quite cognizant of this sort of thing.May also be some de-obfuscation tools out there, albeit for ad hoc cases... dunno. JavaScript comes to mind, widely being delivered in compacted, minified, obfuscated formats. Not that that's necessarily applicable to C syntax, so would likely take a lot of work to adapt.Tim

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 09:46:29 am
content: Quote from: RoGeorge on October 29, 2021, 09:44:48 amummm, Microsoft distributes a compiled binary on code dot visualstudio dot com, it's called "Multiplatform Visual Studio Code from Microsoft (binary version)" aka "vscode" (downloadable fromhereandhere), it has some source to be compiled, but it also looks that the core is released as a binary-blob only available forx86-64 -> VSCode-linux-x64amd64 -> VSCode-linux-x64arm -> VSCode-linux-armhfarm64 -> VSCode-linux-arm64When they say "multiplatform", they mean that you can download it for {Windows, macOS, Linux} on Visual Studio Code's website, but talking about the supported Linux architectures, well, the bin package only works with {x86/64bit, arm/32bit(hardfloat(1)), arm64}Technically it could run on aAllwinner SoC(1)) , but but unfortunately I'm with a PPC64 machine; I mean ... I could instantiate aQemu/arm(1) emulator just to runvscode, but well, it doesn't sound a neat solution.There is also a repositoryhere, and that's what Microsoft says:QuoteAnd this is about the compiled binary that they distribute on code dot visualstudio dot comQuoteThat sounds that it's virtually possible but not easy to compile it for my PPC64 machine

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 09:50:22 am
content: Yup, by Googling I see it's been attempted before, but also it looks like there are still some messy details to work out.Possible, but not a piece of cake.

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 09:56:19 am
content: has anyone triedAnjuta?it's nice and easy to be compiled, the only problematic dependency is the infamousgnome-libgdalibrary that is still buggy and requires you to configure and build with "introspection" enable otherwise it failsQuote(nobody knows why, nobody, not even science)

author: PlainName
author_rank: Super Contributor
timestamp: October 30, 2021, 10:35:40 am
content: QuoteWhy are you one PPC64? I thought you were developing for that, but assumed you had a 'normal' x86 PC to actually do work on.

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 11:03:52 am
content: Quote from: dunkemhigh on October 30, 2021, 10:35:40 amLong history, mainly used for research.

author: nfmax
author_rank: Super Contributor
timestamp: October 30, 2021, 11:07:34 am
content: Can you not edit/refactor your code on an x86 computer, then ship the source over to your PPC for compilation?

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 12:16:47 pm
content: Quote from: nfmax on October 30, 2021, 11:07:34 amI don't have x86 hardware on hands.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 30, 2021, 04:51:17 pm
content: As said already, most decent programming editors have that feature.That said, and probably just like you, I don't even get how anyone could write a 25 kLOC project with such a poor coding style? (And judging from just this variable naming thing, I'm pretty sure this is not the only atrocious side of the coding....) How is it possible to even work on it for more than a couple weeks?

author: PlainName
author_rank: Super Contributor
timestamp: October 30, 2021, 05:09:24 pm
content: QuotePerhaps it wasn't like that originally but got obfuscated deliberately.

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 05:19:02 pm
content: Quote from: dunkemhigh on October 30, 2021, 05:09:24 pmYou. Probably, or auto-generated by other tools and merged together

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 05:22:53 pm
content: Quote from: SiliconWizard on October 30, 2021, 04:51:17 pm25KLOC processed in 294 seconds by my tool to have decent variable and function names2 hours to prepare the tool, 5 months to write the library used by tool, that's computer science

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 05:34:16 pm
content: There are also 2193 "labels", and 911 "goto"This can be fixed by no tool, I can't, you can't, nobody can ...Certain parts of the source look a dump from tools like "IDA", I wonder is it a kind of reverse engineering from assembly-to-C? Who knows

author: harerod
author_rank: Frequent Contributor
timestamp: October 30, 2021, 05:36:38 pm
content: I haven't seen Eclipse mentioned, so far. I am not familiar with pure Eclipse, but CubeIDE has "refactoring" tools, which include variable renaming.That being said - seeing those code snippets catapulted me back some forty years, when single letter variable names were a thing. Thanks for sharing, I envy you not.

author: cfbsoftware
author_rank: Regular Contributor
timestamp: October 30, 2021, 09:25:23 pm
content: Quote from: DiTBho on October 30, 2021, 05:34:16 pmWhy don't you try poking your eye with a sharp stick? It should be a more pleasurable way of spending your time

author: Nominal Animal
author_rank: Super Contributor
timestamp: October 30, 2021, 10:09:10 pm
content: Quote from: cfbsoftware on October 30, 2021, 09:25:23 pmI'm not DiTBho, but I have this horribly annoying flaw that I tend to say "No problem!" when I mean "I can't give any guarantees, but heck, I'll have a go at it!".More than once, this has lead me to do stuff that was less fun than sanding my eyeballs with sharp shards of glass.What I have learned, is how useful documentation describing the overall purpose and algorithms implemented is.Instead of killing oneself slowly by trying to work with such codebases, one can just rewrite the whole mess, preferably fixing any inefficient/unwanted behaviour at the same time.My own code, I've learned to write in modules, mostly so that I can replace/rewrite/reuse the useful bits, without doing everything from scratch. By "modules", I mean in adaptable units, as compared to "fixed" libraries. Nowadays, I experiment with a lot of algorithms and possible approaches to answering interesting questions by writing the core part in such a module, then a set of unit tests to verify its behaviour.. and only if successful, move the module into library form for others to play with. I often do 2-3 before I find the approach whose efficiency and behaviour I'm happy with. I find it fun; I'd love to do that as a job.

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 10:18:52 pm
content: Quote from: Nominal Animal on October 30, 2021, 10:09:10 pmYup, that's the point. It's not my code, the customer asked for a refactoring, that's what I am paid for.Personally I would rewrite the whole code, but there is a problem: there is no documentation at all, the code itself is the documentation, and ... even it somehow works, even if nobody knows *how* and *why*.So, let's acquire the documentation, and let's the customer understand what to ask next: rewrite? Ok, let's talk about business.It looks like a fair plan. I don't want to force anyone.

author: DiTBho
author_rank: Super Contributor
timestamp: October 30, 2021, 10:23:48 pm
content: Quote from: harerod on October 30, 2021, 05:36:38 pmYes, I have a virtual machine prepared by Wind River. Has Eclipse loaded and pre configured. As soon as I have an x86 laptop, I'll give it a try. Judging by its "readme.txt" it looks very powerful and reach of features.

author: rsjsouza
author_rank: Super Contributor
timestamp: October 30, 2021, 11:05:40 pm
content: Quote from: dunkemhigh on October 30, 2021, 05:09:24 pmJob security. I have seen this before. Quite a few times.Quote from: harerod on October 30, 2021, 05:36:38 pmI was going to mention Eclipse as well, since it has good refactoring tools, OSS and it is multi plataform, however DitBho' lib ssems to have solved the issue already.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 31, 2021, 02:35:02 am
content: Quote from: DiTBho on October 30, 2021, 05:19:02 pmDon't underestimate how nasty some developers can be...But yeah, it might just be auto-generated code. If it is, then refactoring it to make it suitable for maintenance by humans is likely a very painful task, with not necessarily a gigantic benefit for future maintenance in the end...