Patent ID: 12214813

DETAILED DESCRIPTION

Various embodiments of a railroad track feature detection system are described according to the present disclosure. It is to be understood, however, that the following explanation is merely exemplary in describing the devices and methods of the present disclosure. Accordingly, several modifications, changes, and substitutions are contemplated.

As shown inFIG.1, a railroad track feature detection system100may include a railroad chassis102. In some embodiments, the railroad chassis102may be associated with an unmanned rail maintenance vehicle (e.g., a drone vehicle), a manned rail maintenance vehicle, and/or another rail vehicle. The railroad chassis102may include wheels104for interfacing with underlying railroad tracks (e.g., rails106). The railroad chassis102may be towed behind another rail vehicle as the rail vehicle propels itself along the rails106of the railroad track. Alternatively, the railroad chassis102may be self propelled and thus may include an engine, a propulsion system, and/or another operating system (not pictured) for propelling the railroad chassis102along the rails106of the railroad track. In further embodiments, the railroad chassis102may take the form of a road-rail chassis or hy-rail vehicle, which may be operated on both roads and rail.

As described throughout, the railroad track may include a pair of elongated, substantially parallel rails106, which may be coupled to a plurality of laterally extending rail ties108. In some embodiments, a top surface of each rail tie108may be coupled to a bottom surface of the rails106. The rail ties108may be disposed on a ballast bed of hard particulate material such as gravel (e.g., ballast, rocks, and/or the like) and may be used to support the rails106.

The railroad track may further include a variety of track features110used for securing the rails106to rail ties108, the ground, and/or other structures. For example, track features110such as spikes or anchors may be used to couple a rail106to an underlying rail tie108. As described herein, a track feature110may refer to any type of railroad track hardware such as an anchor, a spike, a rail tie, a tie plate, a tie hole, a rail joint, a switch, a rail hole, and/or the like.

Additionally, the rails106may include a variety of visual markers112that are disposed on a surface of the rail106. These markers112may be drawn or printed onto a surface of the rail106(e.g., an inside surface, an outer surface, a top surface, and/or the like) using paint, an adhesive, lasers, and/or other means. The markers112may be utilized to identify the locations of track features110along the rails106. For example, a marker112may be disposed on an inside surface of a rail106at a location immediately adjacent to a track feature110(e.g., a rail tie or an anchor). Different color markers112may be used to denote different types of track features110and/or different sections of railroad tracks. For example, blue paint markers may be used to identify anchor locations, whereas green markers may be used to identify rail joint locations. In some embodiments, the markers112may even be painted or printed directly onto track features110.

The railroad track detection system100may further include an enclosure114for housing various electronic components. For example, the enclosure114may house a circuit board or motherboard for executing various operations described herein. In some embodiments, the contents of the enclosure114may be pressurized so that dust, vegetation, and other unwanted particulate may not enter the enclosure. As such, a low-pressure compressor116may be operatively coupled to the enclosure114. The compressor116and/or the enclosure114may include one or more pneumatic valves for ensuring a proper seal for maintaining a desired pressurization of the enclosure114.

The enclosure114may be coupled to the railroad chassis102via a mounting bracket118. The mounting bracket118may enable removable coupling of the enclosure114and the railroad chassis102so that the positioning of the enclosure114relative to the railroad chassis102is configurable. In some embodiments, the enclosure114may be coupled to an underside of the railroad chassis102. Alternatively, the enclosure114may be coupled to a side surface of the railroad chassis102.

The track feature detection system100may also include one or more cameras120. As seen inFIG.1, a camera120may be operatively coupled to the enclosure114via a mounting bracket122. In some embodiments, the mounting bracket122may be pivotable so that the camera120may be positioned at an angle relative to the ground. The mounting bracket122may also be telescopic, mechanized, and/or include one or more joints for precisely positioning the camera120.

The camera120may be utilized to capture images of the rails106as the railroad chassis102is propelled along the rails106. For example, the camera120may be positioned in such a way as to capture images of an inside surface of the rail106. In some embodiments, the camera120may be configured to capture a video feed and/or still image frames of the rails106. The camera120may include visual sensors, a high-speed camera, a color camera, a global shutter camera, and/or the like. The camera120may be enabled to capture high-resolution images of approximately 640 pixels by 480 pixels or higher and at high speeds (e.g., approximately 12 frames per second to approximately 30 frames per second or higher). Additionally, the camera120and/or various sensors may also measure a height, a width, an orientation, a shape, a contour, an angle, a condition, and/or other factors associated with the rails106, track features110, and/or color markers112.

To assist the camera120with capturing high-quality images, a light124may be utilized by the track feature detection system100. The light124may be operatively coupled to the enclosure114and/or the camera120via a mounting bracket122. In some embodiments, the light124may be positioned adjacent to the camera120. For example, the light124may be positioned such that a longitudinal axis of light being projected from the light124may be substantially parallel (e.g., on a common plane) as an axis associated with a lens of the camera120. In other embodiments, the light124may be positioned opposite the camera120and/or in such a way that a longitudinal axis of light being projected from the light124is angled relative to an axis associated with a lens of the camera120.

The light124may include a high-intensity light such as a light emitting diode (LED) direct light, a laser, and/or the like. In some embodiments, the light124may include multiple lights positioned in different positions around the enclosure114and/or camera120.

As the camera120captures images of the rail106, the light124may be used to illuminate various track features110and/or markers112on the rails106. For example, light emitted by the light124may cause track features110such as anchors or spikes to cast shadows on the surface of the rail106. These shadows may be detected in captured images to identify track features110at various locations of the rails106. For example, a track feature110that produces one shadow of a particular length at a location on the rail may be identified in a captured image as a spike, whereas a track feature110that produces four shadows of a particular length and at a different location on the rail may be identified in a captured image as an anchor.

Additionally, color markers112may be illuminated by the light124so that a color or design of each marker112may be accurately identified in captured rail images. The light124may also be helpful in differentiating between track features110, color markers112, shadows, vegetation, and/or stains (e.g., oils, hydraulic fluids, grease, and/or the like) in captured images. The camera120and/or the light124may be calibrated so as to set a frame of capture that enables accurate identification of objects (e.g., track features110, color markers112, shadows, vegetation, stains, and/or the like) in captured images. For example, an image frame edge and/or contour of the camera120may be offset and/or shifted so that clear images of the rail106, track features110, and/or markers112may be captured. In some embodiments, a bottom edge of the camera's120range of detection may be aligned with a foot of the rail106. As another example, the intensity and/or direction of the light124may be modified so as to reduce unwanted glare and/or adjust for various weather conditions and/or amounts of dust, particulate, and/or vegetation.

Once captured by the camera120, images of the rails106may be sent from the camera120to the enclosure114for processing and/or storage. Various computer hardware described herein may be included in the enclosure114. Additionally and/or alternatively, captured images may be sent from the camera120and/or the enclosure to another computing device for processing and/or storage. For example, images captured by the camera120may be transmitted by a communication device included in the enclosure114to an off-site and/or cloud-based computing system for processing and/or storage. In some embodiments, captured images may be transmitted by components included in the enclosure114as they are received from the camera120(e.g., in real time). Alternatively, captured images may be transmitted at predetermined intervals, when the railroad chassis102is determined to be within proximity to a particular communication network, and/or at another time.

Location information associated with each captured image may also be collected by components included in the enclosure114. This location information may be used to identify a location on the rails106at which a track feature110, a color marker112, and/or another object is detected. Location information may include GPS coordinates, X-Y coordinates, and/or other information and may be transmitted with the captured images for processing and/or analysis.

Captured images may be analyzed by computing components as described herein for the purpose of identifying track features110in the images. Identified track features110may be tagged with identifiers and corresponding location information in a database. In this manner, the locations of various track features110on the rails106may be tracked and monitored for maintenance purposes.

Various computing techniques may be deployed when performing image processing on captured images. For example, techniques such as pixel color identification and analysis, support vector machine, edge detection, histograms of gradients, background subtraction, hue/saturation/value (HSV) values, red-green-blue (RGB) color values, pattern detection, and/or other image processing techniques may be used by one or more computing processors as described herein. Scales, angles of capture, length of shutter, shutter speed, brightness, darkness, contrast, hue, saturation, and/or a variety of other characteristics may be set (e.g., selected and/or weighted) by a user or automatically by the railroad track feature detection system100to ensure accurate and efficient capture and processing of images.

The railroad track feature detection system100may identify patterns of track features110and/or associated shadows, markers112as well as location information of the same to determine which identifier is to be assigned to an identified object in a captured image. Additionally, the railroad track feature detection system100may determine a railroad track layout of the rails106based on identified track features110.

Ultimately, the track feature detection system100may enable more efficient documentation and tracking of existing rails106and track features110. Maps and models of the rails106may be generated by the track feature detection system100, where track features110and/or color markers112are displayed digitally. Recommendations for performing maintenance on various track features110may be generated without requiring human inspection. For example, identifying a marker of a particular color that is associated with a particular rail tie may cause the track feature detection system100to recommend a maintenance vehicle to pull (e.g., replace) that particular rail tie. As such, maintenance of the rails106and associated track features110may be streamlined in an automated and cost-effective manner.

FIG.2illustrates a computing system200for enabling the railroad track feature detection system100ofFIG.1and/or other railroad track feature detection systems described herein. In some embodiments, the computing system200may include a first server202and a second server204.

The first server202and the second server204may be communicatively coupled to one another by a network206as described herein. As such, a variety of data may be transmitted between the first server202and the second server204during image capture and analysis processes described herein. The network206may include any wireless and/or wired communications network that facilitates communication between the first server202and the second server204, as well as between any other computing devices (e.g., a user's smartphone, a third party server, and/or the like). For example, the network206may include an Ethernet network, a cellular network, a computer network, the Internet, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a Bluetooth network, a radio frequency identification (RFID) network, a near-field communication (NFC) network, a laser-based network, and/or the like. In some embodiments, the network206may also include a plurality of networks.

The first server202may be associated with the railcar chassis102ofFIG.1and may be housed in the enclosure114. For example, the first server202may include a computer onboard the railroad chassis102that enables operation of electronics included in the enclosure114, the compressor116, the mounting bracket118, the camera120, the mounting brackets122, and/or the light124ofFIG.1. In some embodiments, the first server202may include a computing device such as a mainframe server, a content server, a communication server, a laptop computer, a desktop computer, a handheld computing device, a smart phone, a smart watch, a wearable device, a touch screen, a biometric device, a video processing device, an audio processing device, a cloud-based computing solution and/or service, and/or the like. The first server202may also include multiple servers configured to communicate with one another and/or implement load-balancing techniques described herein.

The first server202may include various elements of a computing environment as described herein (e.g., computing environment300ofFIGS.3A and3B). For example, the first server202may include a processing unit208, a memory unit210, an input/output (I/O) unit212, and/or a communication unit214. Each of the processing unit208, the memory unit210, the input/output (I/O) unit212, and/or the communication unit214may include one or more subunits and/or other computing instances as described herein for performing operations associated with track feature detection.

The second server204may be associated with an off-site computing device for processing captured images of railroad tracks. For example, the second server204may include a cloud-based server that is not onboard the railroad chassis102ofFIG.1. In some embodiments, the second server204may include a computing device such as a mainframe server, a content server, a communication server, a laptop computer, a desktop computer, a handheld computing device, a smart phone, a smart watch, a wearable device, a touch screen, a biometric device, a video processing device, an audio processing device, a cloud-based computing solution and/or service, and/or the like. The second server204may also include multiple servers configured to communicate with one another and/or implement load-balancing techniques described herein.

The second server204may include various elements of a computing environment as described herein (e.g., computing environment300ofFIG.3AandFIG.3B). For example, the second server204may include a processing unit216, a memory unit218, an input/output (I/O) unit220, and/or a communication unit222. Each of the processing unit216, the memory unit218, the input/output (I/O) unit220, and/or the communication unit222may include one or more subunits and/or other computing instances as described herein for performing operations associated with track feature detection.

FIG.3AandFIG.3Billustrate exemplary functional and system diagrams of a computing environment300for enabling railroad track feature detection and associated image processing techniques described herein. Specifically,FIG.3Aprovides a functional block diagram of the computing environment300, whereasFIG.3Bprovides a detailed system diagram of the computing environment300Additionally, any units and/or subunits described herein with reference to the computing environment300ofFIG.3Aand/orFIG.3Bmay be included in one or more elements ofFIG.2, such as the first server202and/or the second server204. The computing environment300and/or any of its units and/or subunits described herein may include general hardware, specifically-purposed hardware, and/or software.

The computing environment300may include, among other elements, a processing unit302, a memory unit304, an input/output (I/O) unit306, and/or a communication unit308. As described herein, each of the processing unit302, the memory unit304, the I/O unit306, and/or the communication unit308may include and/or refer to a plurality of respective units, subunits, and/or elements. Furthermore, each of the processing unit302, the memory unit304, the I/O unit306, and/or the communication unit308may be operatively and/or otherwise communicatively coupled with each other so as to facilitate railroad track feature detection and associated image analysis techniques described herein. Further, the processing unit302, the memory unit304, the I/O unit306, and/or the communication unit308may refer to the processing unit208, the memory unit210, the I/O unit212, and/or the communication unit214of the first server202ofFIG.2. Additionally, the processing unit302, the memory unit304, the I/O unit306, and/or the communication unit308may refer to the processing unit216, the memory unit218, the I/O unit220, and/or the communication unit222of the second server202ofFIG.2.

The processing unit302may control any of the one or more units304,306,308, as well as any included subunits, elements, components, devices, and/or functions performed by the units304,306,308included in the computing environment300. The described sub-elements of the computing environment300may also be included in similar fashion in any of the other units and/or devices included in the system200ofFIG.2. Additionally, any actions described herein as being performed by a processor may be taken by the processing unit302alone and/or by the processing unit228in conjunction with one or more additional processors, units, subunits, elements, components, devices, and/or the like. Additionally, while only one processing unit302may be shown inFIG.3Aand/orFIG.3B, multiple processing units may be present and/or otherwise included in the computing environment300or elsewhere in the overall system (e.g., railroad track feature detection system200ofFIG.2). Thus, while instructions may be described as being executed by the processing unit302(and/or various subunits of the processing unit302), the instructions may be executed simultaneously, serially, and/or otherwise by one or multiple processing units.

In some embodiments, the processing unit302may be implemented as one or more computer processing unit (CPU) chips and/or graphical processing unit (GPU) chips and may include a hardware device capable of executing computer instructions. The processing unit302may execute instructions, codes, computer programs, and/or scripts. The instructions, codes, computer programs, and/or scripts may be received from and/or stored in the memory unit304, the I/O unit306, the communication unit308, subunits and/or elements of the aforementioned units, other devices and/or computing environments, and/or the like.

In some embodiments, the processing unit302may include, among other elements, subunits such as a profile management unit310, a content management unit312, a location determination unit314, a graphical processing unit (GPU)316, a visual recognition unit318, a tagging unit320, a tracking unit322, and/or a resource allocation unit324. Each of the aforementioned subunits of the processing unit302may be communicatively and/or otherwise operably coupled with each other.

The profile management unit310may facilitate generation, modification, analysis, transmission, and/or presentation of a profile associated with a railroad track. For example, the profile management unit310may operate a database associated with a railroad track and may manage information associated with the railroad track and/or a railroad chassis as described herein. The profile management unit310may also control and/or utilize an element of the I/O unit232to enable the capture and/or storage of images of a railroad track. The profile management unit310may receive, process, analyze, organize, and/or otherwise transform any data received from the user and/or another computing element so as to generate a profile of a railroad track that includes route information, status information, track feature information, maintenance information, marker information (e.g., color values), and/or location information of the same.

The content management unit312may facilitate generation, modification, analysis, transmission, and/or presentation of user interfaces for controlling the railroad track feature detection system described herein. For example, the content management unit312may control the audio-visual environment and/or appearance of application data during execution of various processes. Media content for which the content management unit312may be responsible may include advertisements, images, text, themes, audio files, video files, documents, and/or the like. In some embodiments, the content management unit312may also interface with a third-party content server and/or memory location.

The location determination unit314may facilitate detection, generation, modification, analysis, transmission, and/or presentation of location information. Location information may include global positioning system (GPS) coordinates, a mile marker, an Internet protocol (IP) address, a media access control (MAC) address, geolocation information, an address, a port number, a zip code, a server number, a proxy name and/or number, device information (e.g., a serial number), and/or the like. In some embodiments, the location determination unit314may include various sensors, a radar, and/or other specifically-purposed hardware elements for enabling the location determination unit314to acquire, measure, and/or otherwise transform location information. In some embodiments, location-acquiring hardware may be included in the enclosure114of the system100ofFIG.1for determining locations of the railroad chassis102, track features110, and/or color markers114during operation.

The GPU unit316may facilitate generation, modification, analysis, processing, transmission, and/or presentation of images of railroad tracks captured by the camera120ofFIG.1. In some embodiments, the GPU unit316may be utilized to render visual content for presentation on a user device, analyze a captured image, and/or the like. The GPU unit316may also include multiple GPUs and therefore may be configured to perform and/or execute multiple processes in parallel.

The visual recognition unit318may facilitate analysis and/or processing of captured images for the purpose of identifying objects associated with a railroad track. For example, the visual recognition unit318may be utilized for identifying track features and/or markers in captured images. In some embodiments, the visual recognition unit318may include GPUs and/or other processing elements so as to enable efficient analysis of images in either series or parallel. The visual recognition unit318may utilize a variety of image analysis techniques such as pixel value determination, pixel value comparison, edge detection, support vector machine, histogram of gradients, background subtraction, HSV, object recognition, pattern detection, shadow recognition, image filtering, video and/or image frame sampling, image splicing, image reconstruction, error detection, video reconstruction, and/or the like to accurately identify objects of interest in captured rail images. The visual recognition unit318may determine a confidence score of how likely an object identified in an image is indeed a track feature or color marker. For example, the determined confidence score associated with an identified object may be required to exceed a threshold in order for an identifier and/or location information may be assigned to the identified object. The visual recognition unit318may also be configured to filter out unwanted objects in a captured image. For example, an oil stain on a rail or vegetation that has been identified in a captured image may be filtered out using one or more image processing techniques. As another example, an object identified in an image that is smaller than a predetermined size (e.g., a number of pixels) may be filtered out of the image and/or otherwise ignored by the visual recognition unit318.

The tagging unit320may facilitate the accurate assignment of object identifiers and/or location information to identified objects in captured rail images. Various identifiers such as metadata, color values, numerical values, and/or other tags may be assigned to identified objects in a database.

The tracking unit322may facilitate the monitoring, analysis, and/or processing of identifiers and/or location information associated with identified objects. The tracking unit322may monitor a status of an identified track feature at a particular location in such a manner that maintenance for that track feature can be recommended. For example, based on when an image of the track feature was last captured and/or a condition of the track feature detected in the image, the tracking unit322may generate an alert or recommendation for manually inspecting the track feature. Alternatively, the tracking unit322may generate instructions for a drone rail maintenance vehicle to repair and/or inspect the track feature.

The resource allocation unit324may facilitate the determination, monitoring, analysis, and/or allocation of computing resources throughout the railroad track feature detection system described herein. For example, the railroad track feature detection system may facilitate a high volume of (e.g., multiple) communication connections between a large number of supported railroad chassis and/or associated processing servers (e.g., the first server202and/or the second server204ofFIG.2). As such, computing resources of the computing environment300(and/or any subunit of the aforementioned units) such as processing power, data storage space, network bandwidth, and/or the like may be in high demand at various times during operation. Accordingly, the resource allocation unit324may be configured to manage the allocation of various computing resources as they are required by particular units and/or subunits of the computing environment300and/or other computing environments. In some embodiments, the resource allocation unit324may include sensors and/or other specially-purposed hardware for monitoring performance of each unit and/or subunit of the computing environment300, as well as hardware for responding to the computing resource needs of each unit and/or subunit. In some embodiments, the resource allocation unit324may utilize computing resources of a second computing environment separate and distinct from the computing environment300to facilitate a desired operation.

For example, the resource allocation unit324may determine a number of simultaneous communication connections and/or incoming requests for data and/or image processing. The resource allocation unit324may then determine that the number of simultaneous communication connections and/or incoming requests for meets and/or exceeds a predetermined threshold value. Based on this determination, the resource allocation unit324may determine an amount of additional computing resources (e.g., processing power, storage space of a particular non-transitory computer-readable memory medium, network bandwidth, and/or the like) required by the processing unit302, the memory unit304, the I/O unit306, the communication unit308, and/or any subunit of the aforementioned units for enabling safe and efficient operation of the computing environment300while supporting the number of simultaneous communication connections and/or incoming requests. The resource allocation unit324may then retrieve, transmit, control, allocate, and/or otherwise distribute determined amount(s) of computing resources to each element (e.g., unit and/or subunit) of the computing environment300and/or another computing environment.

In some embodiments, factors affecting the allocation of computing resources by the resource allocation unit324may include the number of ongoing communication connections and/or other communication channel connections, a number of image analysis and/or reporting processes, a duration of time during which computing resources are required by one or more elements of the computing environment300, and/or the like. In some embodiments, computing resources may be allocated to and/or distributed amongst a plurality of second computing environments included in the computing environment300based on one or more factors mentioned above. In some embodiments, the allocation of computing resources of the resource allocation unit324may include the resource allocation unit324flipping a switch, adjusting processing power, adjusting memory size, partitioning a memory element, transmitting data, controlling one or more input and/or output devices, modifying various communication protocols, and/or the like. In some embodiments, the resource allocation unit324may facilitate utilization of parallel processing techniques such as dedicating a plurality of GPUs included in the processing unit302for processing a high-quality video stream of a captured rail images.

In some embodiments, the memory unit304may be utilized for storing, recalling, receiving, transmitting, and/or accessing various files and/or information during operation of the computing environment300. For example, the memory unit304may be utilized for storing image frames, video streams, storing, recalling, and/or updating railroad track profile information, marker information, track feature information, and/or the like. The memory unit304may include various types of data storage media such as solid state storage media, hard disk storage media, and/or the like. The memory unit304may include dedicated hardware elements such as hard drives and/or servers, as well as software elements such as cloud-based storage drives. For example, the memory unit304may include various subunits such as an operating system unit326, an application data unit328, an application programming interface (API) unit330, a profile storage unit332, a content storage unit334, an object storage unit336, a secure enclave338, and/or a cache storage unit340.

The memory unit304and/or any of its subunits described herein may include random access memory (RAM), read only memory (ROM), and/or various forms of secondary storage. RAM may be used to store volatile data and/or to store instructions that may be executed by the processing unit302. For example, the data stored may be a command, a current operating state of the computing environment300, an intended operating state of the computing environment300, and/or the like. As a further example, data stored in the memory unit304may include instructions related to various methods and/or functionalities described herein. ROM may be a non-volatile memory device that may have a smaller memory capacity than the memory capacity of a secondary storage. ROM may be used to store instructions and/or data that may be read during execution of computer instructions. In some embodiments, access to both RAM and ROM may be faster than access to secondary storage. Secondary storage may be comprised of one or more disk drives and/or tape drives and may be used for non-volatile storage of data or as an over-flow data storage device if RAM is not large enough to hold all working data. Secondary storage may be used to store programs that may be loaded into RAM when such programs are selected for execution. In some embodiments, the memory unit304may include one or more databases for storing any data described herein. Additionally or alternatively, one or more secondary databases located remotely from the computing environment300may be utilized and/or accessed by the memory unit304.

The operating system unit326may facilitate deployment, storage, access, execution, and/or utilization of an operating system utilized by the computing environment300and/or any other computing environment described herein. In some embodiments, the operating system may include various hardware and/or software elements that serve as a structural framework for enabling the processing unit302to execute various railroad track feature detection and/or image processing operations described herein. The operating system unit326may further store various pieces of information and/or data associated with operation of the operating system and/or the computing environment300as a whole, such as a status of computing resources (e.g., processing power, memory availability, resource utilization, and/or the like), runtime information, modules to direct execution of operations described herein, and/or the like.

The application data unit328may facilitate deployment, storage, access, execution, and/or utilization of an application utilized by the computing environment300and/or any other computing environment described herein (e.g., the first server202and/or the second server204ofFIG.2). For example, users may be required to download, access, and/or otherwise utilize a software application on a computing device in order for various operations described herein to be performed. As such, the application data unit328may store any information and/or data associated with the application. Information included in the application data unit328may enable a user and/or computer processor to execute various operations described herein. The application data unit328may further store various pieces of information and/or data associated with operation of the application and/or the computing environment300as a whole, such as a status of computing resources (e.g., processing power, memory availability, resource utilization, and/or the like), runtime information, modules to direct execution of operations described herein, and/or the like.

The API unit300may facilitate deployment, storage, access, execution, and/or utilization of information associated with APIs of the computing environment300and/or any other computing environment described herein. For example, the computing environment300may include one or more APIs for enabling various devices, applications, and/or computing environments to communicate with each other and/or utilize the same data. Accordingly, the API unit330may include API databases containing information that may be accessed and/or utilized by applications and/or operating systems of other devices and/or computing environments. In some embodiments, each API database may be associated with a customized physical circuit included in the memory unit304and/or the API unit330. Additionally, each API database may be public and/or private, and so authentication credentials may be required to access information in an API database.

The profile storage unit332may facilitate deployment, storage, access, and/or utilization of information associated with profiles of railroad tracks, track features, and/or markers. For example, the profile storage unit332may store a track profile, identification information, marker information, track feature information, location information, and/or metadata associated with a railroad track and/or a railroad chassis. In some embodiments, the profile storage unit332may communicate with the profile management unit310to receive and/or transmit information associated with a railroad profile.

The content storage unit334may facilitate deployment, storage, access, and/or utilization of information associated with requested content by the computing environment300and/or any other computing environment described herein. For example, the content storage unit334may store one or more user interfaces, application information, captured images of rails, video feeds of rails, processed image frames, audio content, and/or metadata to be presented to a user and/or otherwise utilized during operations described herein. In some embodiments, the content storage unit334may communicate with the content management unit312to receive and/or transmit content files.

The object storage unit336may facilitate deployment, storage, access, analysis, and/or utilization of information associated with objects identified in image. For example, the object storage unit336may store location information and/or identifiers associated with objects identified in captured image frames. In some embodiments, the object storage unit336may communicate with the GPUs316, the visual recognition unit318, the tagging unit320, and/or the tracking unit322to facilitate analysis of any stored image frame and/or associated information.

The secure enclave338may facilitate secure storage of data. In some embodiments, the secure enclave338may include a partitioned portion of storage media included in the memory unit304that is protected by various security measures. For example, the secure enclave338may be hardware secured. In other embodiments, the secure enclave338may include one or more firewalls, encryption mechanisms, and/or other security-based protocols. Authentication credentials of a user may be required prior to providing the user access to data stored within the secure enclave338.

The cache storage unit340may facilitate short-term deployment, storage, access, analysis, and/or utilization of data. In some embodiments, the cache storage unit340may serve as a short-term storage location for data so that the data stored in the cache storage unit340may be accessed quickly. In some embodiments, the cache storage unit340may include RAM and/or other storage media types that enable quick recall of stored data. The cache storage unit340may include a partitioned portion of storage media included in the memory unit304.

The I/O unit306may include hardware and/or software elements for enabling the computing environment300to receive, transmit, and/or present information. For example, elements of the I/O unit306may be used to capture images of railroad tracks, display images of railroad tracks, receive user input from a user via a user device, present information to a user, and/or the like. In this manner, the I/O unit306may enable the computing environment300to interface with a railroad track and/or a human user. As described herein, the I/O unit306may include subunits such as an I/O device342, an I/O calibration unit344, and/or video driver346.

The I/O device342may facilitate the receipt, transmission, processing, presentation, display, input, and/or output of information as a result of executed processes described herein. In some embodiments, the I/O device342may include a plurality of I/O devices. For example, the I/O device342may include a variety of elements that enable capturing of images of railroad tracks such as a camera, a sensor, a light, and/or the like. The I/O device342may also include hardware for interfacing with a user, such as a keyboard, a touchscreen, a button, a sensor, a biometric scanner, a laser, a microphone, a camera, and/or another element for receiving and/or collecting input from a user. Additionally and/or alternatively, the I/O device342may include a display, a screen, a sensor, a vibration mechanism, a light emitting diode (LED), a speaker, a radio frequency identification (RFID) scanner, and/or another element for presenting and/or otherwise outputting data to a user. In some embodiments, the I/O device342may communicate with one or more elements of the processing unit302and/or the memory unit304to execute operations described herein.

The I/O calibration unit344may facilitate the calibration of the I/O device342. For example, the I/O calibration unit344may detect and/or determine one or more settings of the I/O device342, and then adjust and/or modify settings so that the I/O device342may operate more efficiently.

In some embodiments, the I/O calibration unit344may utilize a driver346(or multiple drivers) to calibrate the I/O device342. For example, a video driver346may be installed on a computer that enables a camera to capture images of railroad tracks in a particular manner. In some embodiments, the I/O device342may be calibrated by the I/O calibration unit344based on information included in the driver346.

The communication unit308may facilitate establishment, maintenance, monitoring, and/or termination of communications (e.g., a communication connection) between computing devices of the railroad track feature detection system described herein. The communication unit308may further enable communication between various elements (e.g., units and/or subunits) of the computing environment300. In some embodiments, the communication unit308may include a network protocol unit348, an API gateway350, an encryption engine352, and/or a communication device354. The communication unit308may include hardware and/or software elements.

The network protocol unit348may facilitate establishment, maintenance, and/or termination of a communication connection between computing environment300and another computing environment (e.g., the first server202and the second server204ofFIG.2) by way of a network. For example, the network protocol unit348may detect and/or define a communication protocol required by a particular network and/or network type. Communication protocols utilized by the network protocol unit348may include Wi-Fi protocols, Li-Fi protocols, cellular data network protocols, Bluetooth® protocols, WiMAX protocols, Ethernet protocols, powerline communication (PLC) protocols, and/or the like. In some embodiments, facilitation of communication between the computing environment300and any other device, as well as any element internal to the computing environment300, may include transforming and/or translating data from being compatible with a first communication protocol to being compatible with a second communication protocol. In some embodiments, the network protocol unit348may determine and/or monitor an amount of data traffic to consequently determine which particular network protocol is to be used for establishing a video communication connection, transmitting data, and/or performing other operations described herein.

The API gateway350may facilitate the enablement of other devices and/or computing environments to access the API unit330of the memory unit304of the computing environment300. For example, a user device may access the API unit330via the API gateway350. In some embodiments, the API gateway350may be required to validate user credentials associated with a user of a user device prior to providing access to the API unit330to the user. The API gateway350may include instructions for enabling the computing environment300to communicate and share information with another device.

The encryption engine352may facilitate translation, encryption, encoding, decryption, and/or decoding of information received, transmitted, and/or stored by the computing environment300. Using the encryption engine, each transmission of data may be encrypted, encoded, and/or translated for security reasons, and any received data may be encrypted, encoded, and/or translated prior to its processing and/or storage. In some embodiments, the encryption engine352may generate an encryption key, an encoding key, a translation key, and/or the like, which may be transmitted along with any data content.

The communication device354may include a variety of hardware and/or software specifically purposed to enable communication between the computing environment300and another device, as well as communication between elements of the computing environment300. In some embodiments, the communication device354may include one or more radio transceivers, chips, analog front end (AFE) units, antennas, processing units, memory, other logic, and/or other components to implement communication protocols (wired or wireless) and related functionality for facilitating communication between the computing environment300and any other device. Additionally and/or alternatively, the communication device354may include a modem, a modem bank, an Ethernet device such as a router or switch, a universal serial bus (USB) interface device, a serial interface, a token ring device, a fiber distributed data interface (FDDI) device, a wireless local area network (WLAN) device and/or device component, a radio transceiver device such as code division multiple access (CDMA) device, a global system for mobile communications (GSM) radio transceiver device, a universal mobile telecommunications system (UMTS) radio transceiver device, a long term evolution (LTE) radio transceiver device, a worldwide interoperability for microwave access (WiMAX) device, and/or another device used for communication purposes.

FIG.4Adepicts an exemplary visual processing400of a captured railroad track image. An image402of a surface of a rail404may be captured by a camera of the railroad track feature detection system. Water406has been poured on the rail404to show how the railroad track feature detection system described herein may differentiate between blemishes and/or stains on the rail404from other objects.

The image402may be taken by the camera in tandem with a light shining on the rail surface. By using a light, track features408may be illuminated for easy visual detection. Additionally, the light allows the track features408to cast shadows410on the surface of the rail404. The light also illuminates markers412painted onto the surface of the rail404so that its exact color may be easily identified. Additional markers414have been superimposed on the image402for simulation.

As seen in processed image416, with all white, color, and/or light removed, a “negative” image is produced. This negative image highlights the shadows410cast by the track features408. In this manner, track features may be identified using shadow detection and/or background subtraction techniques. Using these techniques, the water stain406and markers412,414are filtered out of the processed image416.

In processed image418, the water406, track features408, shadows410, and marker412have been filtered out. The processed image418shows how different color markers may be identified by the railroad track feature detection system. A range of colors can be detected in captured images. Markers of a color that falls within a desired range may appear more solid, like the rightmost marker414in the processed image418, than markers of a color that fall outside a desired range (e.g., marker412). In some embodiments, each color may be assigned a range of hue values associated with a color. For example, on a color wheel with 360 degrees of colors, a first color may be assigned to degrees 1-10, whereas a second color may be assigned to degrees 11-20, and so on. The railroad track feature detection system may be configured to search captured images for a particular color and/or color range. For example, the railroad track feature detection system may search captured images for colors associated with a hue within the range of degrees 1-20. In this manner, HSV processing may identify color markers. The railroad track feature detection system may also cycle through multiple hue and/or color ranges at a particular location so that markers of multiple colors may be identified. As described above, different color markers may be associated with different track features.

Additionally, shadows may be utilized to identify track features. For example, a number of shadows, a size or length of a shadow, a location of a shadow, and/or other characteristics may be used to identify track features.

FIG.4Bdepicts an exemplary visual processing420of a captured railroad track image. Similar to above, captured image422shows superimposed markers426of various colors. Processed image424shows how markers of some colors are filtered out using HSV processing, whereas markers of other colors are easily identified. When searching for markers of a particular color, the railroad track feature detection system may search based on hue values only (e.g., saturation, value, brightness, transparency, and/or the like may be set to zero). Alternatively, a variety of values associated with a color may be utilized for searching for markers of particular colors. Using edge detection (e.g., at locations where neighboring pixels have color and/or hue values that exceed a predetermined threshold), the railroad track feature detection system may identify a contour, a moment, and/or outline of a track feature, a shadow, a marker, and/or another object.

FIG.4Cdepicts an exemplary visual processing426of a captured railroad track image. Similar to above, captured image428illustrates an image of a surface of a rail430with two track features432, each casting a shadow434on the surface of the rail430. The rail430also includes a water stain436and a grease stain438.

Processed image440shows how the grease stain438may be incorrectly identified as a track feature shadow434by the railroad track feature detection system. Error detection processes may be utilized to eliminate errors. For example, the grease stain438may be filtered out from an image frame based on its size. If the grease stain438(or another object that should not be identified as a track feature and/or marker) is smaller than a predetermined threshold size, then the railroad track feature detection system may filter out the grease stain438.

It is to be appreciated that the processed images according to the present disclosure can identify various anchor types, such as Ericson, stead, unit, woodings and improved fair anchor types.FIG.5Aillustrates an exemplary captured image500and an associated processed image502of a first anchor type504according to the present disclosure. The boxes in the processed image502indicate an identified track feature (e.g., an anchor). Other track features shown in the captured image500, such as spikes506, may also be identified in the processed image502.

FIG.5Billustrates an exemplary captured image510and an associated processed image512of a second anchor type514according to the present disclosure. The box in the processed image512indicates an identified track feature (e.g., an anchor). Other track features shown in the captured image510, such as spikes516, may also be identified in the processed image512.

FIG.5Cillustrates an exemplary captured image520and an associated processed image522of a third anchor type524according to the present disclosure. The boxes in the processed image522indicate an identified track feature (e.g., an anchor). Other track features shown in the captured image520, such as spikes526, may also be identified in the processed image522.

FIG.5Dillustrates an exemplary captured image530and an associated processed image532of a fourth anchor type534according to the present disclosure. The red boxes in the processed image532indicate an identified track feature (e.g., an anchor). Other track features shown in the captured image530, such as spikes536, may also be identified in the processed image532.

FIG.5Eillustrates an exemplary captured image540and an associated processed image542of two different anchor types504and534as well as pulled spike546according to the present disclosure. The boxes in the processed image542indicate identified track features (e.g., an anchor and spike).

While various implementations in accordance with the disclosed principles have been described above, it should be understood that they have been presented by way of example only, and are not limiting. Thus, the breadth and scope of the implementations should not be limited by any of the above-described exemplary implementations, but should be defined only in accordance with the claims and their equivalents issuing from this disclosure. Furthermore, the above advantages and features are provided in described implementations, but shall not limit the application of such issued claims to processes and structures accomplishing any or all of the above advantages.