Write%20a%20longform%20article%20that%20adheres%20to%20the%20style%20rules%20mentioned%20below.%0A%0A%23%20Title%20and%20Subtitle%0A-%20Choose%20a%20title%20that%20frames%20a%20common%20problem%20provocatively.%0A-%20Use%20a%20subtitle%20that%20presents%20a%20counterintuitive%20twist%20to%20a%20common%20assumption.%0A%0A%23%20Structure%20of%20the%20Article%20(these%20aren't%20section%20names,%20don't%20use%20it%20as%20section%20names)%0A%0A1.%20**Introduction**%20%20%0A%20%20%20-%20Start%20with%20a%20conversational%20hook%20that%20introduces%20the%20topic.%20%0A%20%20%20-%20Draw%20the%20reader%20in%20with%20a%20relatable%20scenario%20or%20a%20surprising%20observation.%0A%20%20%20-%20Use%20short%20sentences%20to%20build%20curiosity%20and%20set%20up%20a%20paradoxical%20question%20or%20concept.%0A%0A2.%20**Establish%20Credibility**%20%20%0A%20%20%20-%20Mention%20your%20professional%20background%20to%20build%20credibility.%0A%20%20%20-%20Describe%20firsthand%20insights%20related%20to%20the%20issue.%0A%0A3.%20**Problem%20Identification%20with%20Human-Focused%20Examples**%20%20%0A%20%20%20-%20Clearly%20outline%20the%20major%20problem.%0A%20%20%20-%20Include%20specific,%20human-centric%20challenges.%0A%20%20%0A4.%20**Direct%20Engagement%20with%20the%20Reader**%20%20%0A%20%20%20-%20Occasionally%20address%20the%20reader%20directly%20to%20encourage%20them%20to%20reflect%20on%20their%20relevant%20life%20experiences.%0A%20%20%0A5.%20**Provide%20Practical%20Examples%20and%20Anecdotes**%20%20%0A%20%20%20-%20Use%20memorable%20and%20relatable%20stories%20to%20illustrate%20main%20points.%0A%20%20%20-%20Focus%20on%20vivid,%20practical%20experiences%20that%20others%20may%20recognize.%0A%0A6.%20**Organizational%20and%20Cultural%20Barriers**%20%20%0A%20%20%20-%20Concentrate%20on%20the%20barriers%20that%20are%20cultural%20or%20organizational,%20rather%20than%20technical%20problems.%0A%0A7.%20**Use%20of%20Contrasting%20Subheadings**%20%20%0A%20%20%20-%20Create%20dynamic%20shifts%20using%20contrasting%20subheadings%20for%20major%20narratives%20in%20the%20article.%0A%0A8.%20**Prompt%20Reflective%20Questions**%20%20%0A%20%20%20-%20Prompt%20readers%20to%20think%20deeply%20using%20an%20open-ended%20question,%20inviting%20them%20to%20reflect.%0A%0A9.%20**End%20with%20a%20Takeaway%20Offering%20a%20New%20Perspective**%20%20%0A%20%20%20-%20Close%20the%20article%20with%20a%20thought-provoking%20insight%20that%20offers%20a%20new%20perspective%20on%20the%20original%20problem%20or%20challenge.%0A%0A%23%20Forbidden%20Words:%0A-%20Do%20not%20use%20the%20following%20words:%20Overall,%20Explore,%20Captivate,%20Tapestry,%20Leverage,%20Embrace,%20Resonate,%20Dynamic,%20Testament,%20Fascinating,%20intersection,%20Delve,%20Elevate,%20Navigate,%20Navigating,%20landscape.%0A%0A%23%20Steps%0A1.%20Write%20a%20title%20that%20is%20provocative%20and%20prompts%20interest.%20Make%20sure%20the%20subtitle%20juxtaposes%20an%20unexpected%20perspective.%0A2.%20Introduce%20the%20topic%20in%20a%20conversational%20way,%20incorporating%20a%20scenario%20or%20striking%20observation%20to%20engage%20the%20reader's%20curiosity.%0A3.%20Establish%20why%20you%20understand%20the%20issue%20firsthand%20by%20giving%20a%20relevant%20background%20detail%20from%20your%20experience.%0A4.%20Outline%20the%20main%20problem%20by%20focusing%20on%20human%20aspects.%20Avoid%20oversimplified%20or%20entirely%20technical%20issues.%0A5.%20Directly%20address%20the%20audience%20occasionally,%20involving%20them%20personally%20with%20experiences%20or%20scenarios.%0A6.%20Use%20anecdotes%20to%20show%20practical%20examples%20that%20give%20depth%20to%20your%20main%20ideas.%0A7.%20Discuss%20obstacles%20that%20are%20based%20on%20organizational%20culture%20or%20dynamics%20rather%20than%20focusing%20on%20a%20lack%20of%20tools%20or%20skills.%0A8.%20Use%20contrasting%20subheadings%20that%20reflect%20two%20aspects%20of%20the%20core%20argument%20to%20arrange%20the%20essential%20thinking.%0A9.%20Include%20a%20reflective,%20thought-provoking%20question%20in%20the%20content%20as%20a%20means%20of%20engaging%20in-depth%20reader%20thought.%0A10.%20End%20with%20a%20perspective-altering%20conclusion%20that%20sums%20up,%20with%20broader%20takeaways%20that%20reflect%20back%20to%20the%20initial%20question.%0A%0A%23%20Output%20Format:%0A-%20**Title**:%20A%20provocative%20main%20title%20with%20a%20subtitle%20that%20offers%20an%20unexpected%20insight.%0A-%20**Body**:%20Long-format%20article,%20consisting%20of%20different%20sections%20from%20Introduction%20to%20Conclusion,%20laid%20out%20per%20the%20rules%20above.%0A%0A%0A%0ATopic%20for%20the%20article:%0A%23%23%23%0A%0AThe%20Polymathic%20Engineer%0A%0A%0A%0AThe%20fallacies%20of%20distributed%20systems%0AEight%20distributed%20systems%20fallacies%20that%20are%20underrated%20during%20system%20design.%0A%0AFRANCO%20FERNANDO%0ANOV%2014,%202024%0A%0A%0A%0A%0AHi%20Friends,%0AWelcome%20to%20the%2097th%20issue%20of%20the%20Polymathic%20Engineer.%20This%20week,%20we%20talk%20about%20the%20fallacies%20of%20distributed%20systems.%0AMore%20than%2020%20years%20ago,%20Peter%20Deutsch%20and%20others%20at%20Sun%20Microsystems%20came%20up%20with%20a%20list%20of%20false%20assumptions%20that%20many%20developers%20new%20to%20distributed%20applications%20always%20make.%0AThese%20assumptions%20are%20always%20shown%20to%20be%20wrong,%20which%20makes%20bugs%20hard%20to%20fix.%20We%20are%20going%20to%20look%20at%20these%20eight%20fallacies%20in%20more%20depth%20here.%0A%0AProject-based%20learning%20is%20the%20best%20way%20to%20build%20technical%20skills.%20CodeCrafters%20is%20an%20excellent%20platform%20to%20practice%20interesting%20projects%20such%20as%20building%20your%20version%20of%20Redis,%20Kafka,%20HTTP%20server,%20SQLLite,%20and%20even%20Git%20from%20scratch.%0ASign%20up,%20and%20become%20a%20better%20software%20engineer.%0A%0AThe%20Network%20is%20Reliable%0AWhen%20building%20a%20distributed%20system,%20it's%20dangerous%20to%20assume%20that%20the%20network%20that%20connects%20your%20components%20works%20perfectly.%0AIndeed,%20networks%20are%20inherently%20unreliable:%20packets%20can%20be%20dropped,%20connections%20can%20be%20interrupted,%20and%20data%20can%20become%20corrupted%20during%20transmission.%20And%20that's%20just%20the%20beginning.%0ATo%20build%20a%20robust%20system,%20we%20need%20to%20accept%20and%20plan%20for%20these%20potential%20failures.%20An%20effective%20strategy%20is%20to%20put%20in%20place%20retransmission%20mechanisms.%20The%20store%20and%20forward%20pattern%20is%20a%20common%20way%20to%20do%20this.%0A%0A%0AInstead%20of%20directly%20sending%20data%20to%20a%20downstream%20server,%20we%20can%20store%20it%20locally%20or%20in%20an%20intermediate%20location.%20This%20allows%20for%20retries%20and%20provides%20some%20ways%20to%20get%20back%20on%20track%20in%20catastrophic%20scenarios%20where%20a%20simple%20retry%20loop%20wouldn't%20work.%0AMessage%20brokers%20like%20RabbitMQ%20and%20ActiveMQ%20are%20ideal%20to%20support%20this%20pattern,%20as%20are%20various%20cloud-native%20solutions%20offered%20by%20major%20providers.%0AFallacy%20%232:%20Latency%20is%20Zero%0ALatency%20is%20the%20time%20it%20takes%20for%20data%20to%20move%20from%20one%20place%20to%20another,%20and%20it%20is%20a%20fact%20you%20can't%20ignore%20in%20distributed%20systems.%0ALatency%20is%20like%20extra%20work%20that%20every%20request%20has%20to%20do.%20Regardless%20the%20messages%20are%20large%20or%20small,%20latency%20remains%20constant%20since%20it%20is%20primarily%20affected%20by%20the%20speed%20of%20light%20and%20the%20distance%20between%20communicating%20systems.%0AThe%20most%20obvious%20way%20to%20mitigate%20latency%20issues%20is%20to%20move%20data%20closer%20to%20clients.%20In%20a%20cloud%20environment,%20we%20can%20choose%20availability%20zones%20carefully%20based%20on%20your%20clients'%20locations.%0AIn%20addition,%20we%20can%20use%20caching%20to%20reduce%20network%20calls%20and%20duplicate%20the%20data%20closer%20to%20where%20it%20is%20needed%20using%20Content%20Delivery%20Networks%20(CDNs).%0AThe%20Polymathic%20Engineer%20is%20a%20reader-supported%20publication.%20To%20receive%20new%20posts%20and%20support%20my%20work,%20consider%20becoming%20a%20free%20or%20paid%20subscriber.%0A%0A%0A%0A%0A%0A%0A%0A%0A%0ASubscribe%0A%0AFallacy%20%233:%20Bandwidth%20is%20Infinite%0ABandwidth%20is%20the%20amount%20of%20data%20that%20can%20be%20transferred%20in%20a%20given%20time%20and%20is%20not%20unlimited.%20While%20bandwidth%20has%20improved%20over%20time,%20the%20amount%20of%20data%20we%20send%20also%20increased.%0AModern%20applications%20like%20VoIP,%20or%20video%20streaming%20consume%20a%20lot%20of%20bandwidth.%20Even%20simple%20things%20like%20richer%20UIs%20and%20verbose%20data%20formats%20(like%20XML)%20increase%20the%20data%20transfer.%0A%0A%0AThere's%20a%20delicate%20balance%20between%20minimizing%20latency%20and%20conserving%20bandwidth.%20We%20should%20transfer%20more%20data%20in%20fewer%20trips%20to%20reduce%20latency.%20However,%20we%20need%20to%20send%20less%20data%20overall%20to%20save%20bandwidth.%0AEfficient%20data%20formats%20and%20compression%20are%20all%20things%20that%20help%20address%20bandwidth%20limitations.%0AFallacy%20%234:%20The%20Network%20is%20Secure%0AAssuming%20the%20network%20we%20are%20using%20is%20secure%20is%20a%20critical%20mistake.%20Security%20should%20be%20a%20top%20priority%20with%20the%20rise%20of%20bug%20bounty%20programs%20and%20frequent%20news%20of%20major%20exploits.%0AMalicious%20users%20constantly%20try%20to%20intercept%20and%20decode%20network%20communications.%20This%20makes%20data%20encryption%20and%20security%20risk%20mitigation%20essential.%0AKeeping%20security%20in%20mind%20from%20the%20start%20of%20system%20design%20always%20pays%20off%20in%20the%20long%20term.%20Also,%20it's%20important%20to%20take%20the%20time%20to%20look%20carefully%20for%20possible%20security%20threats.%0AFallacy%20%235:%20Topology%20Doesn't%20Change%0AThe%20network%20structure%20isn't%20static.%20It%20changes%20frequently,%20sometimes%20due%20to%20something%20breaking,%20but%20most%20of%20the%20time%20because%20we%20add%20or%20remove%20components%20on%20purpose.%0A%0A%0AWith%20the%20rise%20of%20cloud%20computing%20and%20containerization,%20network%20topology%20changes%20are%20even%20more%20common.%20Also,%20elastic%20scaling,%20where%20we%20add%20or%20remove%20servers%20based%20on%20workload,%20requires%20flexibility.%0AZookeeper%20and%20Consul%20are%20great%20tools%20that%20help%20solve%20problems%20with%20service%20discovery%20and%20let%20apps%20adapt%20to%20changes%20in%20how%20our%20systems%20are%20set%20up%20and%20how%20they%20work.%0AFallacy%20%236:%20There%20is%20One%20Administrator%0AAs%20a%20system%20grows,%20it%20often%20relies%20on%20external%20systems%20outside%20our%20control.%20This%20means%20we%20can't%20manage%20everything%20ourselves.%0AIt's%20crucial%20to%20consider%20all%20the%20dependencies,%20from%20our%20code%20to%20the%20servers%20we%20run%20on.%20As%20the%20number%20of%20systems%20and%20configurations%20increases,%20it%20becomes%20hard%20to%20manage%20and%20track%20everything.%0AThe%20first%20way%20is%20to%20implement%20robust%20monitoring%20and%20observability%20tools,%20which%20are%20critical%20for%20diagnosing%20issues%20quickly%20when%20they%20arise.%0AAlso,%20using%20Infrastructure%20as%20Code%20(IaC)%20to%20codify%20the%20system%20variations%20and%20focus%20on%20appropriate%20decoupling%20helps%20ensure%20overall%20system%20resiliency%20and%20uptime.%0AFallacy%20%237:%20Transport%20Cost%20is%20Zero%0ASending%20data%20between%20systems%20is%20a%20simple%20business%20cost%20only%20when%20things%20are%20small.%20However,%20as%20systems%20grow,%20the%20data%20transport%20costs%20become%20more%20significant.%0AAt%20a%20certain%20point,%20we%20may%20think%20it%20be%20worth%20optimizing.%20For%20example,%20message%20formats%20like%20JSON%20can%20be%20heavier%20than%20transfer-optimized%20formats%20like%20gRPC.%0AIn%20cloud%20environments%20like%20AWS,%20data%20transfer%20costs%20between%20regions%20or%20availability%20zones%20are%20real%20and%20can%20add%20up%20quickly.%0ABeing%20aware%20of%20these%20costs%20is%20essential.%20However,%20it%20has%20its%20tradeoffs.%20If%20we%20optimize%20too%20early,%20it%20might%20cause%20us%20more%20trouble%20than%20it's%20worth%20in%20the%20short%20run.%0AFallacy%20%238:%20The%20Network%20is%20Homogeneous%0AWhile%20we%20like%20everything%20to%20be%20clean%20and%20tidy,%20the%20real%20world%20is%20far%20from%20it.%20As%20an%20example,%20we%20often%20need%20to%20transform%20one%20data%20format%20into%20another.%0AIt%20is%20important%20to%20be%20compatible.%20Because%20of%20this,%20our%20systems%20will%20still%20work%20even%20if%20a%20new%20framework%20comes%20out%20or%20we%20need%20to%20use%20them%20in%20places%20they%20weren't%20designed%20for.%0AInteroperability%20does%20have%20some%20limits,%20though.%20We%20need%20to%20find%20the%20right%20balance.%0AWe%20can%20save%20time%20and%20trouble%20in%20the%20long%20run%20if%20we%20keep%20in%20mind%20that%20not%20all%20systems%20are%20the%20same%20and%20avoid%20coupling%20our%20solution%20to%20one%20specific%20aspect.%0AInteresting%20Read%0ASome%20cool%20articles%20I%20read%20the%20past%20week:%0A%09%E2%80%A2%09Horizontal%20Scaling%20Starts%20with%20Load%20Balancers%20%E2%80%93%20But%20Doesn%E2%80%99t%20Stop%20There%20by%20%E2%80%A8%E2%80%A8Raul%20Junco%20%0A%09%E2%80%A2%09Intro%20to%20GraphQL%20by%20%E2%80%A8Saurabh%20Dashora%20%0A%09%E2%80%A2%09Why%20%22bias%20for%20action%22%20is%20not%20good%20enough%20by%20%E2%80%A8Irina%20Stanescu%20%0A%09%E2%80%A2%09The%20$15%20Investment%20That%20Could%20Save%20Your%20Developer%20Eyes%20in%202025%20(And%20It's%20Not%20Another%20Blue%20Light%20Filter)%20by%20%E2%80%A8Akos%20%0A%0A%0A%0A%0A%0ADiscussion%20about%20this%20post%0AComments%0ARestacks%0A%0A%0A%0A%0A%0A%0ARaul%20Junco%0ASystem%20Design%20Classroom%0ANov%2014%0A%0ALiked%20by%20Franco%20Fernando%0AI%20learned%20some%20of%20these%20the%20hard%20way!%0AThanks%20for%20the%20mention,%20Fernando.%0A%0AREPLY%0A%0ASHARE%0A%0A%0A%0A%0A%0ASaurabh%20Dashora%0ASystem%20Design%20Codex%0ANov%2018%0ANot%20being%20aware%20of%20these%20fallacies%20is%20a%20sure%20way%20of%20running%20into%20trouble%20when%20working%20with%20distributed%20systems.%0AThanks%20for%20the%20mention%20as%20well,%20Fernando!%0A%0AREPLY%0A%0ASHARE%0A%0A%0A2%20more%20comments...%0A%0ATop%0ALatest%0ADiscussions%0A%0ANo%20posts%0A%0AReady%20for%20more%3F%0A%0A%0A%0A%0A%0A%0A%0A%0A%0ASubscribe%0A%0A%0A%C2%A9%202024%20Franco%20Fernando%0APrivacy%20%E2%88%99%20Terms%20%E2%88%99%20Collection%20notice%0A%0AStart%20WritingGet%20the%20app%0ASubstack%20is%20the%20home%20for%20great%20culture%0A%0A%0A%0A%23%23%23