Code Reuse (Big company style) - Page 1
Forum section: electronics / general

author: metrologist
author_rank: Super Contributor
timestamp: March 07, 2017, 09:28:00 pm
content: I'm wondering what facilitates code reuse, and what does code reuse mean, exactly? I am imagining a programmer wanting to write a function, say it is something simple like metric conversion. Someone probably already wrote a function like this. What facilitates the programmer to find this and reuse it?Does reuse mean the programmer can just point to the existing function, so that if the function was ever modified (bug fix), that the fix would automatically proliferate to all uses of the code? Is the code copied and disconnected from the original source?What about partial functions or other code bits, can they be used in parts in an efficient way? How is it managed and used?I'm thinking of in large commercial industries, such as Keysight for example. I am sure there are things like graphing functions that are reused in their equipment and other features. It seems that at a high level, a complete modular software component can be reused in whole. That is more analogous to each one of us having our own browser instance installed, and not what I think of if discussing code reuse.

author: DimitriP
author_rank: Super Contributor
timestamp: March 07, 2017, 11:42:21 pm
content: y ! LQQK here ?https://en.wikipedia.org/wiki/Reusability

author: metrologist
author_rank: Super Contributor
timestamp: March 08, 2017, 12:16:56 am
content: It's an article that says a whole lot of nothing. Levitation is a concept I can understand. One could identify levitation and say their company uses levitation to increase efficiency. I want to levitate. How to levitate? I can describe the mechanics of walking, including effects of gravity, the fall, leg and foot movements to accomplish the motion. Levitation? It's a unicorn to me and all terms are meaningless abstracts.For example, the article tells me things that I've read before:"Because reuse implies the creation of a separately maintained version of the assets, it is preferred over leverage."This makes no sense to me. If an asset is a separately maintained version of some other asset, then you have two separate assets and have merely duplicated and modified, not reused. Leverage is a term that just seems a bit semantic."Reusability brings several aspects to software development that do not need to be considered when reusability is not required."A big, Huh? I cannot convert that into something meaningful for me."Software reusability more specifically refers to design features of a software element (or collection of software elements) that enhance its suitability for reuse."It's designed to work.I still know nothing about how reuse works in any meaning full way. I have written an Arduino function that takes some data and converts it into a 40-bit word. I have another function that takes that word and streams it over the ISP buss. In the sea of millions of .ino files I have, how would I ever find these code bits to reuse, or remember or know they even exist? What facilitates this knowledge and guides a programmer, and how does it work in practice?It would seem there is an extension to intellisense, where if I started writing a 40-bit word conversion function, something would alert me that this code already exists. How is that? Big companies have millions or billions of lines of code. How is it managed and what is the user (programmer) experience or workflow and process?I've never seen this. Maybe it is one of those controversial things in the programming world?

author: helius
author_rank: Super Contributor
timestamp: March 08, 2017, 12:38:47 am
content: In "The Mythical Man-Month", Fred Brooks (the manager of the OS/360 project at IBM) discusses four kinds of software program.The first, is simply a program. He assigns it a difficulty weighting of 1.The second is a softwareproduct. As a product, it must be properly documented and supported. This has a difficulty of 3.The third issystem software. This needs to be carefully designed to run with privileges. Difficulty 3.The fourth and last is asystem software product. By inheriting the requirements of the second and third types, it has a difficulty of 9.You've observed that the language used by enterprise software consultants is feckless and vague, but that's entirely expected because it comes from enterprise software consultants. Never listen to men in suits.Is "code reuse" a good thing? I would say no. The cost of extracting functionality into a library, documenting, and supporting it as a library will always be higher. It suffers from a tendency to overdesign because it's not being written with a specific purpose in mind, but a spectrum of predicted purposes. It is also more resistant to change: if you discover a bug in a library, you can't always fix it because other software depends on how it works now. Better to know patterns and use them, not libraries.

author: cyberfish
author_rank: Regular Contributor
timestamp: March 08, 2017, 01:52:27 am
content: Quote from: metrologist on March 07, 2017, 09:28:00 pmI'm not sure if you mean in the embedded world specifically... I don't have professional embedded development experience unfortunately, but I have worked at a few big companies (currently at Google) in software development.Code reuse is mostly in the form of libraries - for example, Google has infrastructure libraries for things like distributed data processing pipelines, distributed databases, distributed and fault-tolerant storage, machine learning, etc. Application developers never have to write that code. We have teams maintaining those libraries, and application developers just use them. This makes a lot of sense, because developing a distribued data processing pipeline for example is a whole lot of work that also requires highly domain-specific knowledge, and many teams need something like that. It wouldn't make sense to duplicate all that effort.There are also things like utility libraries that are a bit like C++'s Boost - small functions to generate random numbers with different distributions, etc.There is no reuse below function level. Code snippets are not reused. Anything that should be reusable needs to go into functions (libraries are simply a collection of functions).

author: JPortici
author_rank: Super Contributor
timestamp: March 08, 2017, 06:23:22 am
content: since there are chunks of data (big functions / blocks) that i reuse from project to project (like the UART comunication protocol, soon to be moved to CAN) the last months have been mainly about rewriting all these functions to be compartimentalized (ugh.. does this word even exist? compartments. i divided code in compartments)so that i can just grab them and copy them into a new project. make some adjustments (like don't get comm data from UART but get it from CAN) and compile.seems standard procedure but it took me some year(s) and a book on c++ to get it

author: metrologist
author_rank: Super Contributor
timestamp: March 08, 2017, 04:56:26 pm
content: Quote from: helius on March 08, 2017, 12:38:47 amLoL - best assessment! I understand the strategy and have better insight to what is happening. Code re-use was not really what I thought. Thanks!I am looking at it more from the tech doc side because there, systems exist that allow small phrases to be wrapped as portable elements, which can be serialized and re-used in part, and the elements are annotated with loads of metadata to help content architects assemble the pieces into something meaningful. Perhaps there is more flexibility there since the written word does not really have to execute! I have no idea how those systems facilitate the actual turning of the nuts and bolts. I saw the potential for code re-use to be treated in a similar way. It seems not at all...

author: hli
author_rank: Frequent Contributor
timestamp: March 08, 2017, 08:10:21 pm
content: As a rule of thumb: you need to use your code / library in at least three different scenarios / contexts to get its programming interface (API) right. Incidentally this corresponds with the needed effort to create a library - if you re-use your code three times you are about to break even.So if its something you already know will be used in different projects / scenarios, its worth to invest from the beginning to write it as a library. Otherwise, just write good, clean code. Create a good API which doesn't make too many assumptions (the rule here is: low coupling but high adhesion). When you then find this code would be a good fit to be used somewhere else, extract it. If you use it a third time, make a library out of it and document it.

author: alexanderbrevig
author_rank: Frequent Contributor
timestamp: March 08, 2017, 08:32:19 pm
content: Code reuse in the small is to organize your own code so you can save time down the line. If you write a driver for a particular peripheral, extract that in a way that makes it easy to include it in the next project. This way you spend time getting X up and running once, if you find a bug down the line - you fix that bug once and it trickles down to the other projects using that code.Code reuse in the big is to have libraries which you link into. Say you're the application developer at some company for getting the Next Big IoT device out the door. Maybe you've got one team working on the radio protocol stack, one team working on an RTOS of some kind. Another team prepares low lever drivers for you. You job is to assemble the actual device and make it behave as they want. In the meantime, another team works on variants of the device, or maybe the gateway.Now, every team is only responsible for their own piece of the universe. They will have a test suite that covers their needs, and the way change trickles down from them is by a new linked library.A bugfix in the RF stack trickles down to all products using it.Code reuse in the big is hard, and few do it right. There is no QED for what is right either - what works is most important.

author: nctnico
author_rank: Super Contributor
timestamp: March 08, 2017, 08:37:03 pm
content: Quote from: metrologist on March 07, 2017, 09:28:00 pmHave a programmers anonymous meeting once every week and tell what you are working on and/or talk to co-workers if they know a solution for a particular problem. The worst thing to do is to chase some holy grail library. I've seen that fail because software just isn't that flexible. Also the API of a library needs to be frozen solid. In a piece of software which is under construction that is never the case.

author: alexanderbrevig
author_rank: Frequent Contributor
timestamp: March 08, 2017, 08:55:10 pm
content: Quote from: nctnico on March 08, 2017, 08:37:03 pmhttps://en.wikipedia.org/wiki/Open/closed_principleAn API is open for additions and closed for removals. A good API describes the interface to that complete universe, if you need to hide something later - you've done something wrong. Then there's the whole deprecation issue and everyone is mad. We all know that f.ex a serial peripheral needs to send and receive, so start there. No problem adding a peek or a flush later. If a project requires a polling driver, then that project can request the library to add it - or simply work around it.Workarounds are a greater threat to 'code reuse' than the potential 'too many access points' problem of an API. If you see a pattern of 'workarounds' then report it and request it as a feature.Stay DRY.In my mind, the best APIs never ever enforces state on me as a client. I want to tell, rather than ask. I definitely don't want to remember.

author: obiwanjacobi
author_rank: Super Contributor
timestamp: March 09, 2017, 11:18:01 am
content: Quote from: alexanderbrevig on March 08, 2017, 08:55:10 pmI have never interpretted the open-closed principle as additions and removal.Depricating stuff is normal. Prediction is very hard, especially where the future is concernedSo the idea that you can have the perfect API from the start is a flawed premise. Evolution is normal and that is also true for libraries and library writers.The key is to be explicit about it and to give the clients/user time to adapt. No changes should be forced upon the clients. Moving to a new version is always the clients choice.API design, version management, dependency management and feature planning are key in maintaining a good library. Not easy, it requires lots of experience to be able to make the right choices but it can be done.When you harvest code from existing projects make sure you remove all dependencies not absolutely necessary and that the code has been used in three different contexts successfully - otherwise you will find that your generic library code is not as generic as you thought. Be very strict.Aim for 100% code coverage on your unit tests (you do unit test, right?).

author: Howardlong
author_rank: Super Contributor
timestamp: March 09, 2017, 11:40:20 am
content: Here's an example of code reuse, and what can go wrong in an extreme case.https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/There are many issues here, including cloud, cross-site dependencies and crappy black box code, more failure modes than you can shake a stick at, but for in this context it's the crappy black box code we're discussing.The code itself is crap because it's inefficient in some cases. Most people will use the code as a black box, but not realise that it'll perform crappily on iterations on large strings. In early unit testing this crap performance won't necessarily be obvious, but on real data volume it could well be a big deal. Rarely do programmers document the non functional limitations of their code.

author: donotdespisethesnake
author_rank: Super Contributor
timestamp: March 09, 2017, 12:08:37 pm
content: Quote from: metrologist on March 08, 2017, 04:56:26 pmI have never heard of anything like that, can you give some examples? The tech authors I have worked with don't have anything like that.QuoteThat is a fundamental difference which probably renders any comparison invalid.It seems you are looking for some nuts and bolts answers, which will be obvious to software people but depend on the particular tools you are using, so generic articles like those on wikipedia are not much use. Most programmers probably have not thought too much about principles of computer science, they just write code. Computer science has been going for decades, and it is hard to summarise in a short post.The simplest form of reuse is "copy and paste". Clearly this has disadvantages, so language designers quickly introduced into the language the concept of subroutines (also called procedures or functions).Then I guess the next method was object code reuse. By creating libraries of pre-compiled functions, these can be reused efficiently.Up to object oriented languages, those were the only two methods of reuse. Managing and indexing functions was done by Mark I brain, and usually documentation (e.g. man pages) in the form of a list of functions, and how they called, used etc.I don't think there have been any mainstream languages which have built in support for managing reuse. Niklaus Wirth introduced the idea of "literate programming", where documentation is interwoven with source code. That idea is widely used in a simple form with doxygen.The next level of reuse came in "object oriented programming" which really just extends the idea of subroutines, but the addition of inheritance is new. Common example being C++. Again, there is nothing in C++ to manage or index objects, apart from comments in the code.So after 50 years of computer science, we still rely on the brain, talking to people, and documents. There was a fad once for CASE (Computer Aided Software Engineers) tools, which would wrap up a lot of software development stuff into an integrated tool, but that never caught on. The best we have is editors with some primitive levels of intellisense, and some integration with a version control tool.Although I learned to program with FORTRAN, Basic, Pascal, assembler, there is actually very little that has changed in the basic process of writing software. The biggest change is the internet and google, which greatly speed up access to information, but that is really quantitive rather than qualitative. Having found a code snippet or library, I still need to adapt it to my environment, debug it, write test case etc.An interesting exercise is to compare software development with hardware development, and consider what the fundamental differences are, and ponder why over 30 years hardware productivity has increased some 100x where software is maybe 5x better.

author: metrologist
author_rank: Super Contributor
timestamp: March 09, 2017, 05:02:45 pm
content: Your tech authors should be aware of structured authoring and leveraging such concepts as DITA.DITA is one popular example, but its specification addresses granularity to topic level. Companies will adopt such a concept and specialize. I have not used these systems in practice, but am aware of them through contact with career writers. One example was a custom system used by Oracle where individual sentences were all serialized.I would think applying these concepts to code would actually be much simpler. Because, while the written word does not have to execute or compile, it still needs to make sense and what qualifies as making sense has such great variability. I can say something in a dozen different ways, but programming is so much more rigid - the rules and structure have much less variability and are much more predictable.Looking at tech docs again. These usually need to be translated into other languages and here is where principles of structured content provides a great payback. Translators use machine translation to a great extent, but the machine is leveraging a database of human translated content (called translation memory (tm)), broken down to the phrase level. Translators use an environment that has 'intellisense' where they can pick the coverage and % language match. They will see color coded text highlights that indicate language match and they can select the machine translation or enter new or modify. Once the translation memory is built up and writers follow consistent terminology and writing style, pure machine translation starts to cover more of the content since the tool will start to find qualified matches for full sentences or paragraphs. Companies usually do not pay for machine translated content, so when the get thousand page API documents translated, they may only need to pay for a small percentage of human translation. That's Trados software as an example, and SDL puts out several such tools for managing content and other assets.Tech doc authoring has improved as much as mechanical design, likely for the same reasons. These new technologies facilitate efficiency. I recall in civil engineering drawing by hand and running blue print machine. 5 years later there was Atlas CAD and I could design post tensioned bridges and abutments 100x faster with fewer people, and virtually no mistakes.

author: helius
author_rank: Super Contributor
timestamp: March 10, 2017, 02:43:42 am
content: Quote from: metrologist on March 09, 2017, 05:02:45 pmIf you really believe that, you need to spend some time with theInternational Obfuscated C Contest. If the structure was predictable, it follows that the meaning of code was necessarily obvious.QuoteAre the translations actually used by human beings, or are they just checking a box? If no one needs to use them for any real purpose, I'm not surprised that machine translation "works".

author: metrologist
author_rank: Super Contributor
timestamp: March 10, 2017, 06:41:59 pm
content: Not to trivialize the complexity of coding, and I'm not a programmer. At best I am accomplished in Arduino.I'm not saying that code can't be made complex, especially on purpose, but that it does not need to be. Code would seem to be limited and predicable the way I see it, for example in C, if you start a line of code with "For" there is a limited number of things that can follow that. In language, there are many more possibilities to complete the phrase "For...". You can further abstract that in code into element variables and rules for combining them. That is defining a rigid code structure. The For statement can require a condition element comprising three more elements of the start, end, and increment condition, for example. After that, you need another element, but this particular element could be one of many, or many of many, and would be managed separately. You may define a dozed constructs of the For statement and no others would be valid for use as a construct to facilitate reuse.The considerations of translation ROI are more removed from the strategies used to create them. It's also not pure machine translation. For example, "sweep the leg" will have a different translation in the RF world than it does in MMA. All of the translations are human translations and are merely computer assisted. I'm not referring to bablefish or google translate, although these systems can make use of translation memory.

author: westfw
author_rank: Super Contributor
timestamp: March 11, 2017, 07:11:23 am
content: QuoteDocumentation. Generality. Specificity. Correctness. Efficiency. Simplicity. Complexity. Culture.Some of those might seem contradictory - Generality and Specificity, for example. Or Simplicity and Complexity.But ... Suppose you are a networking company with millions of lines of source code, among which is a "getbuffer()" function that gets a memory buffer for a network packet. Simple enough, in principle; obvious, even. So it scores high in Simplicity. And it's specific to the rest of the SW environment for this particular vendor; there's probably "stuff" in the data structure for recording statistics, debugging, different fields for different interfaces, etc. High points for Specificity. But it's also general - you want it to work for IP packets, and XNS packets, and ethernet and SLIP and PPP and 1822... And in order to run as fast as possible, we're going to get these buffers from pre-allocated pools of "common" sized packets. 64byte minimum-ethernet/ping/TCP ACK packets. 576 byte "Maximum minimum MTU recomendation" packets. 1500 byte full ethernet packets. 10000 byte EGP routing updates (no. 12000. I mean 15000. 18000!) And these buffer pools need replenished, and debugged, and monitored... So we're quickly reaching an amount of complexity in the implementation that "clearly has value" without interfering too much with the simplicity of use. Yea! We should re-used our getbuffer() function for ever and ever!(meanwhile, documentation. Code comments; programmer's manual; API reference manual. end-User documentation describing all the bits that are user-visible. And code reviews where if you do something OTHER than use getbuffer() you get a "Why the frick aren't you using getbuffer here?!")QuoteOh, I don't know. There is "refactoring." Get a company of a particular size with the right "culture", and code snippets that are repeated often enough in similar circumstances, and eventually they'll be "refactored" into library functions. Things that are hard to do and cause bugs or annoy people will be re-written. Things that are too slow will be re-written. Things that are done slightly differently in a bunch of different places will get their code snippets "unified" into something better. If you're lucky.