Photogrammatry/Visual SLAM - Page 1
Forum section: electronics / general

author: cdev
author_rank: Super Contributor
timestamp: October 20, 2018, 03:27:16 am
content: Is anybody doing it? I want to map out my small space so I can redecorate better, Using Photogrametry SW like Colmap or VSFM and generating a dense point cloud in MVS eventually bring the dense point cloud into Blender.Anybody else doing this?

author: Siwastaja
author_rank: Super Contributor
timestamp: October 20, 2018, 12:07:46 pm
content: Quote from: cdev on October 20, 2018, 03:27:16 amNot photogrammetry, but I'm working on 3D time-of-flight measurements, tightly integrating everything together for a lower-cost solution (although it still cannot compete with a $5 camera module, or two $5 cameras comprising a stereo camera), while adding a few unique and novel "tricks" to circumvent the biggest accuracy and reliability issues associated with the continuous-wave 3DTOF. The intended application is indoor/outdoor SLAM and obstacle avoidance with about 10 meters of range.You will hear more about this when we have something to actually put in production, but this is looking very good and I believe it is a game changer as a kind of "solid state LIDAR" as some people call it. As for the progress status, we are working on the mass production calibration setup, and on the other hand, on the calibration / data enhancement algorithms. Still, the great thing here is that unlike photogrammetry, time-of-flight sensors are not fundamentally tied to perfect, complex algorithms; we have the basis on direct distance measurement based on the speed of light, which only needs refinement and qualification, and some sensor fusion on software, making the algorithms simpler and less critical.

author: janoc
author_rank: Super Contributor
timestamp: October 20, 2018, 12:47:57 pm
content: Quote from: cdev on October 20, 2018, 03:27:16 amI was working on similar things, what do you want to know?Generally speaking, SLAM is not really what you want for this - I have yet to find a free/low cost application that would do any kind of reasonable job here. Most of them fail badly when they don't see enough texture (i.e. blank walls!) and will get confused by windows.The tracking also drifts - if you stand in the middle of the room and turn 360 the generated map will not fuse properly, you will have a large discontinuity/step between the beginning and end of the scan. The accuracy sucks - the best we have got from a room mapping using these was about 5-10cm, but most often much worse than that. That was both using phone/tablet cameras and dedicated hardware sensors such as the Google Tango or the Structure Sensor.Also, what these generate is a point cloud which is not all that useful in Blender because it is literally just a bunch of points which is far from being sufficiently dense to allow for accurate measurements. The reconstructed meshes tend to be universally useless - the applications offering this are typically meant for 3d scanning of small objects and fail horribly when trying to scan a room. The automatically generated meshes are usually too rough/inaccurate and very polygon-heavy, making them very difficult to use for anything.Photogrammetry works generally better as long as you calibrate your camera and are really careful about how you take the photos/video. However, taking good enough pictures inside of a small room can be a challenge unless you have a fisheye lens (both expensive and more difficult to work with during the reconstruction), arrange sufficient lighting, etc. And it also fails if the walls don't have sufficient texture on them because there aren't enough features for the algorithm to fuse the images.If all you want is data for remodeling your home, you would be far better off with a measuring tape or a laser/ultrasonic range finder. Both are far more accurate and the measurements go actually faster than messing with any kind of scanner. Most rooms are rectangular, so you need only a few distances measured to get the size of the space. Scanning of an even small room takes 10-15 minutes at least, plus any setup and teardown (e.g. covering up windows, placing lights, etc.) and then you need all that post-processing to actually extract the data - if you even manage to do so in the first place and don't have to go back and re-scan.Of course, if you have access to a proper commercial laser scanner that would be different matter. But those are a totally different league, both when capabilities and price are concerned.

author: legacy
author_rank: Super Contributor
timestamp: October 20, 2018, 01:05:58 pm
content: Quote from: Siwastaja on October 20, 2018, 12:07:46 pmeh, talking about business, a customer (for whom I am the consultant) is making a lot of money by providing three 3D-LIDARs as sort of high tech parking sensors for military helicopters.They are in some way rival to SICK, which is a leader in the field of laser scanners, but they have added to the recipe a very intriguing recertification for avionics, for which SICK doesn't seem to have interest.Anyway, these three 3D-LIDARs have been requalified as Aeronautical Radio INC devices to the MTC (mission tactical computer), hence they are connected to the navigation system to which they provides the contribute of being accurately and reliably able to detect and measure objects in good time and in multiple dimensions, and by collecting large volumes of data on multiple scan layers and from different angles, they can detect and respond to objects under the helicopter, like high voltage wires suspended on poles (extremely dangerous if the helicopter accidentally comes in touch with), as well as objects around that are obstructing the path.There are four lasers with different wavelengths in the range of infrared, and photo arrays, and an intriguing use of RANSEC's algorithms (this is the part where I am involved) for using the LIDARs to detect objects that are not visible due to the fog or smoke.I am not an expert of LIDARs, but as far as I have understood, basically, except for the algorithms part, they haven't invented anything, they have taken a technology similar to what is used by SICK and requalified it for being avionics compliant.so, it would be intriguing to see where and who will use "solid state LIDAR" and for what. I know, I am too involved with weird customers, and for sure the micro-robotics is interested

author: cdev
author_rank: Super Contributor
timestamp: October 20, 2018, 01:35:45 pm
content: All of you, thank you very much for your info!This is an interesting area.I've been plodding through the literature on SLAM, 3d reconstruction, 'visual odometry' etc.Since we're on the subject, I've noticed that Kinect cameras are fairly cheap, and there seems to be a software library for it that runs on Linux, freenect.Alternatively, it would be possible to make a rig that held two decent but cheap USB webcams a known distance apart, in a rigid manner, kind of like the eyes. It should be possible to calibrate that.Some really cheap (under $5) USB webcams are surprisingly decent.I have a longstanding interest in archaeology so all this is something that I had been following a bit in the past, but stopped following it for several years and in that time the use of it and applications that use it have really exploded.I've had some success, and as my current requirements are quite modest compared to others, I am confident that I should be able to get this worked out.These programs need to do a lot of calculation and they need lots of memory and lots of disk space to do their jobs, especially when you have a bunch of images. (even though I am taking the images with the smallest resolution I can on my current camera, its not a smart phone camera, they are slightly bigger than that.)As part of this project Ive downloaded at least 15 or 20 SLAM/photogrammetry related software projects and attempted to find a combination that I was able to successfully compile that was also stable which would let me go from a series of photos to a dense point cloud suitable for cleanup and importation into 3D programs via Meshlab. (which is available as a deb and is stable.)It would be much easier if I was using Cuda, and not AMD/OpenCL, as at least for now, that seems to be where most of the software is using. That said, I would hate to spend so much more for the same computer power. And the open software ideal is where I would like to go. So I bought an AMD card.I can get a sparse point cloud from colmap. And VSFM works for me - although some of the programs it uses are not compiling for me right now, I have binaries which I can sort of hack to make them work, but its ugly. (would much rather compile current versions that ran normally) Still, I have several programs on my list of tools to try which I have not dug into in any depth yet, because I have not been able to successfully compile their dependencies. And some I have been able to compile, but which I have issues with, probably because some obvious thing I am not understanding. I.e. user error.Right now I am trying MVS, Ive also been able to successfully compile MVE and micmac, when I leave out the QT user interface.And SiftGPU which is working well in OpenGL mode.You know, maybe I should just bite the bullet and get a second, small, cheap Nvidia card, or maybe a used better card, on ebay, in the past I have seen them go for really cheap, say $50 - just for this stuff.Maybe one with passive cooling that didn't use a lot of power, so I didn't have to worry about additional current used up and heat. So, not so powerful.That would let me use a lot more software.I am taking the pictures with a good camera and a prime lens thats been well characterized, with bright, even lighting, trying to work my way around so each picture contains a lot of common points with the previous one.Also, the OS too is perhaps problematic. If I was on Windows, it would be a lot easier to find a working quick solution, especially if I was willing to buy it, but I really prefer Linux so much more, well, to be frank, the issues i have with Windows are a real joy killer for me.All that said, it seems that I am very close to having a working setup.Janoc, here is what I think I am going to do for flat featureless walls. temporarily put up some targets printed on paper which I have made which will add some detail, basically a 10 cm square with a cross in it. Masking tape doesn't stain the walls. I'll only need to do it once and I can also use tape measurements to get the size exactly right. I think that doing this I should be able to get a better 3D model than I could do myself with the only just beginner Blender modeling skills that I have. (Its only been quite recently that I have been able to get accelerated 3D Blender to run on my machine, which makes a huge huge difference in the learning curve, I realize, so I am really just a beginner.)I have to say, so much of the cool software in this area seems to come from France.My goal is to create a 3D model of my entire interior. This will be especially useful in the work areas, in getting things right in a fairly small space.Lidar is really amazing. I live in an area fairly close to where a large hurricane (Hurricane Sandy) did a lot of damage several years ago, and there was a project done in some places there to see if it could be used to very rapidly assess the damage done after this massive storm came in and knocked down or seriously damaged a great many houses along the shore.They were able to do a rapid scan of houses by driving through these areas and do a very high quality rapid damage assessment based on that 3D data.

author: lukier
author_rank: Unknown
timestamp: October 20, 2018, 02:29:11 pm
content: If you want to scan rooms and get dense reconstruction you need at least semi-dense if not fully dense SLAM. Because of the sheer amount of data beefy GPU is pretty much a must, especially for "real-time" operation.I wouldn't recommend passive mono/stereo, like DTAM:https://www.robots.ox.ac.uk/~vgg/rg/papers/newcombe_davison__2011__dtam.pdfWay too much compute and often poor results in practice, lighting changes, textureless-areas etc.Depth cameras help here a lot (small scale, indoors). One approach is the classic KinectFusion, but because of the way the map is represented you might hit space size / resolution limits pretty quickly:https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ismar2011.pdfElasticFusion is a bit better, as it uses a cloud of surfels to represent the map, it also has loop closures:http://www.roboticsproceedings.org/rss11/p01.pdfcode here:https://github.com/mp3guy/ElasticFusion/(needs CUDA and OpenGL)Meshlab can load these surfels maps and one can use Poisson surface reconstruction there to remesh the cloud into a fully dense, albeit sometimes too smooth model.On the downside the tracking is not that great and as janoc said, as with most SLAM/VO systems it will drift. Loop closure might correct some drift, but not always.The perfect solution is to have an external system for tracking and just use ElasticFusion for mapping. I did that back in the day to obtain full 3D room models for ground truth as the lab was equipped with VICON tracker (sub-mm accuracy), see figure 18:https://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/dyson-robotics-lab/lukierski_etal_icra2017.pdf(it is a can of worms on its own, calibration from VICON to Kinect, synchronising the sensors etc).Other models from real rooms were taken with just ElasticFusion and it's ICP+Lukas-Kanade built-in tracking.LIDARs like Velodyne is a completely different league, but also orders of magnitude higher price (than PC with nVidia GPU + Kinect).I cannot stress the importance of geometric camera calibration for any kind of SLAM enough. For dense systems even more as they rely on pixel data, therefore photommetric calibration should be done as well. Here's an example of SLAM system that does that:https://jakobengel.github.io/pdf/DSO.pdf

author: legacy
author_rank: Super Contributor
timestamp: October 20, 2018, 02:30:38 pm
content: Quote from: cdev on October 20, 2018, 01:35:45 pmhave you ever tried or considered theJK1kit?

author: lukier
author_rank: Unknown
timestamp: October 20, 2018, 02:33:32 pm
content: Quote from: legacy on October 20, 2018, 02:30:38 pmTK1 is long in the tooth and obsoleted by nvidia AFAIR since CUDA 7.0 (they went 64bit only). TX1/TX2 are much better, but even more expensive. For the price of either one can buy quite beefy GPU second hand.

author: janoc
author_rank: Super Contributor
timestamp: October 20, 2018, 03:11:28 pm
content: Quote from: cdev on October 20, 2018, 01:35:45 pmActually nope. Yes, you could get pre-compiled code. But that's all. Building anything computer vision-related on Windows is an horrendously uphill battle because most computer vision people use Linux for development due to its better development tools and libraries. Just try to compile things like OpenCV or PCL in Windows and you will see what it takes ...Quote from: cdev on October 20, 2018, 01:35:45 pmSorry but that's nowhere near enough unless you plaster the entire wall with these markers. Worse, they need to be distinct from each other because otherwise the SLAM algorithm gets confused. E.g. if all the corners of your markers look the same because they come from black squares on a white background (or your crosses) and there are no other features around, it is completely useless - the algorithms using features such as SIFT or ORB won't be able to distinguish between them and will get lost. Basically what works best is something that is almost random, with no apparent structure and repetition in it.Quote from: cdev on October 20, 2018, 01:35:45 pmSeriously, this is a completely ridiculous way of trying to build a model of a room. That's a bit like needing to travel to Tokio and you started with chopping wood to build a plane while trying to learn aeronautical engineering from a textbook because you don't want to learn how to buy a plane ticket.Even if you manage somehow to obtain a scan, the 3D scans arenotusable directly. So you will need to learn to use Blender (or whatever other tool) to actually rebuild the 3D models from them. There is no magic solution where you just take a bunch of pictures and a ready to go 3D model comes out. Not even when using the expensive commercial tools (regardless of what their marketing brochures show).Also, if you don't have a lot of RAM and a powerful GPU, don't even bother - all these algorithms need both in order to get the processing done in a reasonable time. Large, dense scans can take many hours to process even on a powerful machine.Quote from: cdev on October 20, 2018, 01:35:45 pmWhy? What is that model going to be useful for (apart from having something to look at) that you couldn't achieve by taking a bunch of measurements and making a plan?Quote from: cdev on October 20, 2018, 01:35:45 pmYes and those devices cost hundreds of thousands of USD ... Also a typical LIDAR like the Velodyne mentioned before doesn't really produce useful scan - it is meant for navigation/collision avoidance, not for dense mapping of the environment. There is also a huge difference in resolution/accuracy - for a navigation system 10cm or even a meter of error is no big deal (and considered "dense" already) but that is an enormous deal if you are actually trying to take accurate measurements for construction purposes.What these guys were likely using were laser scanners, such as machines from FARO. Those can scan huge areas and go down to millimeters of accuracy. Just if you need to ask about the price you can't afford it.When it comes to SLAM you need to understand that while SLAM stands for "simultaneous localization and mapping", the emphasis is onlocalization. The mapping part is often only for the machine to be able to "find itself" in the space by comparing what it "sees" right now with what it has seen in the past, not to produce an usable 3D model. That's the case of most of the published SLAM algorithms, such as the mentioned DTAM, Monoslam, PTAM, etc. Also keep in mind that these are research systems, often unable to properly do loop closure (to recognize that it has returned to a point it has been to before and fuse the data), don't manage memory in a meaningful way so they will be limited to short runs, etc.Someone has also mentioned visual odometry - that is actually SLAM without building any map at all. The advantage is that it is much faster and needs much less memory, the disadvantage is that once it "gets lost", it has no way to recover. This is e.g. what Apple has in their ARKit.Systems that focus on actual model production work differently because the emphasis is not real time performance but resolution and accuracy. The working principle is roughly the same but the actual math and the way the processing is done is often very different.

author: legacy
author_rank: Super Contributor
timestamp: October 20, 2018, 03:19:46 pm
content: Quote from: lukier on October 20, 2018, 02:33:32 pmumm, not sure about this. I have seen on eBay USA and eBay German a couple of TK1s second hand for ~70 euro each, the original price was 250TX2 is ~500-600 Euro.

author: lukier
author_rank: Unknown
timestamp: October 20, 2018, 03:58:17 pm
content: Quote from: legacy on October 20, 2018, 03:19:46 pmThat is precisely why it is cheap second hand, it is not supported anymore and not very fast (compared to TX1/TX2 not to mention Xavier).

author: lukier
author_rank: Unknown
timestamp: October 20, 2018, 04:06:59 pm
content: Quote from: janoc on October 20, 2018, 03:11:28 pmThat is not entirely true, any (sparse in this context) SLAM system worth its salt does feature matching not only by descriptor distance (that could be fooled by similar corners), but also pixel distance from landmark reprojection, then various triangulation tests and then RANSAC based filtering.Most simple camera calibration tools use a chessboard pattern and most of the time there is no problem with assigning the point correspondences because there is prior knowledge (of the pattern that is).Quote from: janoc on October 20, 2018, 03:11:28 pmIt depends, it is important to understand the differences between systems. Monoslam and PTAM are sparse systems and the map is quite useless, but the tracking is better. Dense systems like DTAM (or ElasticFusion) have rather poor tracking, almost as an afterthought, and focus more on mapping. That's why coupling ElasticFusion with external VICON tracker can get sub-cm model accuracy (if I calibrated Kinect depth - flat field correction and offset/scale it could be even better perhaps).For more thorough explanation and analysis of sparse vs dense and direct vs indirect I recommend Jakob Engel's PhD thesis:https://jakobengel.github.io/pdf/JakobEngelPhD.pdf

author: janoc
author_rank: Super Contributor
timestamp: October 20, 2018, 06:47:18 pm
content: Quote from: lukier on October 20, 2018, 04:06:59 pmI didn't want to go to such detail. However, you can do as much reprojection and RANSAC as you want if all (or a majority) of features seen in the current frame were incorrectly identified. Reprojection and RANSAC will only eliminate the incorrectly matched features so that the algorithm doesn't try to estimate the pose from bad data (which would make it diverge) but it will not "invent" new correct features where there aren't any.At that point the algorithm is basically doing dead reckoning and is only accumulating uncertainty. And after a while of this the tracking will be lost because even if there are enough visible features it won't be able to match them properly anymore.Some systems are capable of performing relocalization (to "find themselves" once they got "lost") from the past data in such situation but that assumes both that the map didn't get corrupted by the bad data (you will only know that the data were bad once the algorithm diverges - but that is too late to reject them from updating the map) and that the pose of the camera is not too different from what was seen before. I have yet to see a SLAM system that is 100% reliable when it comes to this. Even something like Hololens can get pretty badly confused in certain situations.Quote from: lukier on October 20, 2018, 04:06:59 pmThat's not quite comparable because that chessboard pattern constrains your search a lot. You don't care about identifying the individual features (corners in this case) from their descriptors, you only need to find them and then you use the known geometry and planarity of the pattern to label them.Quote from: lukier on October 20, 2018, 04:06:59 pmYes, of course. But that's hardly SLAM anymore and the setup/calibration are quite non-trivial. Also an external tracker like that is not exactly a cheap option. Vicon starts at around $20k for the basic set. Even a cheap Optitrak starts at about $10k or so.It could probably be done using the Valve's Lighthouse system (that the HTC Vive HMD uses), that is relatively cheap (~$300 for 2 bases + one tracking "puck"). On the other hand, I am not sure about how something like Kinect would handle the intense IR flashes and lasers of the tracking system.

author: cdev
author_rank: Super Contributor
timestamp: October 20, 2018, 09:45:01 pm
content: Using RTKlib inPPPmoderelativeGPS accuracy outdoors typically becomes accurate to sub-decimeter even with an imperfect sky view (PPP is a no base station required, one receiver standalone method, that uses one or multiple frequency GPS, usually plus SBAS however this is assuming a very long warmup period, you cant just turn it on from a cold start and expect to get that.) When given short baseline correction data (RTK) then the accuracy becomes fairly absolute, not relative and even indoors its my experience that the accuracy is surprisingly good. (wood frame building.) This wouldnt hold true for any kind of large concrete building though. However where I live, the GPS in RTK mode is perfectly capable of localizing me inside the house to an accuracy of a meter or less, usually better, but only after some time- after its had a long time of continuous operation to warm up (many hours). Actually, that warmup is not necessary if you use AGPS to fetch up to date iono data, the missing part - the reason it takes so long is the extra precise ionospheric correction data, which can be fetched via the network too (phone-collected GPS data has access to post processing using that data) Also, cell network cells now collect DGPS data. Sometimes the baseline involved there can be very short. So increasingly, as more and more cell handsets support raw data its going to be increasingly accurate from the very beginning if they can collect raw data and whomever is using that info has realtime access to that cell site data stream especially.Anyway, the point I am trying to make is that when you combine GPS data with these other methods, the chance of being able to accurately localize something (to just a few centimeters or even less) using a combination of methods that includes SLAM - especially if you combine other methods to eliminate ambiguous situations seems pretty good unless they are deep inside a substantial building, (how about a building whose walls ceilings and floors are either totally flat and featureless, or mirrors? that would be interesting!)But if you use dead reckoning or something like that too, physical odometry, plus one of those electronic gyroscope devices that can keep track of rotation...I bet navigation inside of buildings will be a solved problem within a few years. Especially if they put visual cues of some kind in rooms or on the floor for the robot to pick up on. (Cheap and simple solution for things like mail delivery and cleaning robots.)What this means is that autonomous delivery will become a reality, even in difficult situations, inside of urban areas and even apartment buildings.Sometimes low-tech solutions work as well or better than high tech ones. Plus they create jobs which are going to become scarcer and scarcer due to automation with terrifying consequences for society.We shouldn't automate everything just for the sake of doing so, I guess is what I am trying to say."Back in the day" - to my recollection this must have been in the late 90s, there was a short lived company "webvan" that tried to do the food delivery thing and at least in my area the word was that it worked well.Two guys who lived in my building - computer people - used webvan. This meant they had a small lockable refrigerator installed in the back of our building on the ground floor. The webvan delivery people had a key (the same key the garbage men, etc. had to get into the common areas of the building) and thats all they needed to get to this refrigerator. So they didn't need to be home to accept deliveries and not have them get snatched by some hungry person.

author: cdev
author_rank: Super Contributor
timestamp: October 20, 2018, 10:44:53 pm
content: Way too much $$$ for me, unfortunately. This is why I dislike Nvidia, they aremuchmore expensive"than I feel they should be"."Harrumph"Sucks that they don't see how this is a big barrier to most people. Or maybe they do.Quote from: legacy on October 20, 2018, 02:30:38 pm

author: Dubbie
author_rank: Unknown
timestamp: October 21, 2018, 12:06:34 am
content: I’ve got direct experience with what you’re trying to do and like janoc above, can tell you it will never work like you are planning.At work we use expensive ($100k+) lidar units from FARO and Leica. While they give really nice accurate and dense point clouds, there is no real way to translate that to a useable mesh. Instead we treat it like a set of measurements and model everything from scratch. For certain detailed objects such as rocks and tree trunks, we can use photogrammetry successfully, however this generates large models that usually need to be cleaned up and processed by a skilled 3D artist in a sculpting application afterwards.There isn’t really a shortcut for this kind of stuff, if there was we would be using it and saving thousands.Learn how to use SketchUp, buy a $5 tape measure and you will get accurate and useful results in a day of work.

author: lukier
author_rank: Unknown
timestamp: October 21, 2018, 10:21:27 am
content: Quote from: janoc on October 20, 2018, 06:47:18 pmBut we are not talking about blank walls, if you have blank walls without any texture visual SLAM would fail, if it also has no structure then even Kinect based SLAM will fail. IMU assisted SLAM will drift massively. What I wanted to say is that some features are better than no features, because that's what cdev wanted to add:Quote from: cdev on October 20, 2018, 01:35:45 pmAnd even if the features are "similar" they are in different spatial (therefore reprojection) locations so some of them won't get mismatched, the ones that will should be filtered out anyway. Therefore adding random markers shouldn't hurt and could always help, you'll need just a handful to estimate the pose (well the more the better, but still). That's contrary to what you've said:Quote from: janoc on October 20, 2018, 03:11:28 pmOne should give as much help to SLAM as possible and practical. Anyway, the proof of the pudding is in the eating, so I made a simple example, ORB-SLAM2 in RGBD mode with Kinect, looking at a chessboard pattern displayed on a monitor. The pattern has a lot of pretty much identical corners, so you would say they are not distinct and the system would get confused. In reality you can see the reconstructed landmarks in the 3D view resembling the pattern structure well, triangulated correctly, and the tracking works without trouble.Of course if the drift is big enough (after 360 deg scan) then the reprojection error matching criteria would fail as well, so loop closure will be difficult. Using wide angle (fisheye) optics might help a bit when doing full 360 deg scan, as there might be more point to "anchor" and therefore less drift and chance of mismatches. Anyway, the more features the better, compared to blank flat white walls.Quote from: janoc on October 20, 2018, 06:47:18 pmNothing is 100% reliable, especially SLAM. Every now and then somebody comes and proclaims "SLAM is a solved problem" and it is somehow still not the case.Quote from: janoc on October 20, 2018, 06:47:18 pmSure, that's not SLAM, but it kind of works OK for the purpose of this thread - room scanning. Still cheaper than Velodyne (not to mention FARO).If going lower cost, personally, I would run ORB SLAM2 in RGBD mode for sparse tracking and something like ElasticFusion for dense mapping. Combining the best of both worlds.Quote from: janoc on October 20, 2018, 06:47:18 pmKinect has quite good pass-band filters and defined pattern structure, so I had no problems interference with stuff like VICON for example. I think Vive was OK as well (didn't play with it much back in the day). The problem is the other way around, VICON sometimes treated Kinect IR projector as a marker and I've heard that Vive doesn't have pass-band filters and gets disturbed by Kinect.Quote from: Dubbie on October 21, 2018, 12:06:34 amFrom cdev's posts I guess that he wants to play with 3D reconstruction more for fun than business, given the reluctance to even buying a consumer-grade Nvidia GPU. In that sense various open source 3D reconstruction/SLAM libraries are a fun way to learn and play.If this was business and real accuracy is needed, cm or better, right angles being square then sure, get FARO or, cheaper Matterport (looks 3 kinects glued together with some software). Dubbie is right, you just get point clouds, denser or sparser but point clouds. It's a long way ahead to practical blueprint like model. There are some meshing algorithms, there are various plane fitting tricks (like Manhattan DTAM does), don't get me even started on deep learning based methods, but ultimately it is still a lot of hand crafting.

author: cdev
author_rank: Super Contributor
timestamp: October 21, 2018, 09:27:25 pm
content: What is the cheapest respectably functional Cuda card for under $200? Is it possible to get one that isn't crippled. (I hear Nvidia cripples the Cuda performance of some of their cards)My card was less than that and it is still the equivalent of a much more expensive Nvidia card in terms of horsepower, (using OpenCL) even though its technology is a few years old.

author: lukier
author_rank: Unknown
timestamp: October 21, 2018, 09:45:44 pm
content: Quote from: cdev on October 21, 2018, 09:27:25 pmThey do in terms of double performance or nowadays the number of TensorCores, but that doesn't matter that much except either scientific simulations or 16 bit floating point deep learning.Under $200 you should be easily able to get something like GTX1060 that has 3.5 TFLOPs of fp32 performance:https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#GeForce_10_seriesMaybe even GTX1070 if you find a good deal.GTX900 or older cards might be cheaper, but are sometimes more power hungry and don't support new CUDA features.

author: cdev
author_rank: Super Contributor
timestamp: October 22, 2018, 03:08:17 am
content: Actually, that is exactly why I would be getting one, that and photogrammetry.I'd really like to spend less though. Maybe a GTX1050 with passive cooling. Planned obsolescence is a crucial issue. Vendors like to arbitrarily cut off support for older cards - figure you only get a few years out of each one in a specialty use like machine learning- and they get away with it.They probably don't even support applications like that on non-pro cards.Also, have any of you run two video cards, from different vendors at the same time?Quote from: lukier on October 21, 2018, 09:45:44 pm

author: janoc
author_rank: Super Contributor
timestamp: October 22, 2018, 09:38:11 am
content: Quote from: cdev on October 22, 2018, 03:08:17 amIf you want the card to reliably die on you, do insist on passive cooling.Modern GPUs are not designed for passive cooling and you won't find a reasonable (i.e. not heavily crippled) one that runs with a passive cooler only. There is simply no way to dissipate that much heat these things generate using only a passive heatsink. Even with fans the graphic cards are notorious for dying because of problems linked to thermal cycles and heat.Also, what you likely don't realize that stuff like CUDA loads the card to the limit, probably even more than playing latest games. Those tend to be limited by the monitor framerate because it makes little sense to render much faster than what the monitor can display. With CUDA style calculations there is no such limit so that card is going full blast all the time.Also you want a card that is not crippled - and then you want to buy a 1050 which is the same chip as a 1060 but with half of the CUDA cores disabled and lower clock speed.Even 1060 is a low end card. Again, this is penny pinching in the wrong place.I am not quite sure what you mean by planned obsolescence - if you are buying into this stuff, you must expect that you will have to replace your hardware every two years or so. A lot of the software simply doesn't run on the older cards. These are not applications designed for compatibility - a lot of this code is research code, the researchers don't care whether or not this runs for everyone. They only need the code to run for them so that they can crunch their data and publish papers.Quote from: cdev on October 22, 2018, 03:08:17 amNo, CUDA and similar stuff runs just fine on GeForce consumer grade cards. You are only not allowed to use them in datacenters professionally (recent license restriction on CUDA) but the hardware runs this just fine. You don't need to buy into Quadro (those are optimized for CAD and visualization) or their Titan (specifically CUDA and machine learning oriented) cards.Quote from: cdev on October 22, 2018, 03:08:17 amThat works fine unless you want to run SLI. Then it is not recommended/supported if the cards are different.Quote from: lukier on October 21, 2018, 09:45:44 pmI wouldn't buy a GTX9xx - they are notorious for their flakiness (black screen, freezes, crashes, etc.) and are much more power hungry (and running hotter) than the newer 10xx series. The price and performance difference between a 970/970Ti and a 1060/1070 is negligible and you get a newer generation card that runs cooler and will be supported for longer.

author: lukier
author_rank: Unknown
timestamp: October 22, 2018, 10:00:15 am
content: Quote from: cdev on October 22, 2018, 03:08:17 amWhat 'that', you mean scientific simulations? Then you'll need Tesla cards for double performance, GTX series have some double but very slow.For machine learning fp32 performance is king, not many people use fp16 (that are used with TensorCores) yet although nvidia promotes it heavily. If you need TensorCores you need at least 20 series or TITAN V (or newest Teslas).Quote from: cdev on October 22, 2018, 03:08:17 amPretty much what janoc said, find best bang per buck, no passive cooling. I suggest you research the GPUs before buying to familiarize yourself.I don't get the planned obsolescence as well, only recently Nvidia dropped support of 500 (Fermi) series, these were released in 2011 - that's 7 years.Quote from: cdev on October 22, 2018, 03:08:17 amLike what? Like ElasticFusion? That should run on 1060 no problem and if you don't like CUDA you can always rewrite the tracking (ICP and LukasKanade) to OpenCL or OpenGL compute shaders.

author: cdev
author_rank: Super Contributor
timestamp: October 22, 2018, 06:47:55 pm
content: My current ('Ellesmere' i.e. Polaris family) card does FP16 (half precision), BTW.Performance is good.If you know of any portable cross-platform benchmark I can run I'd be happy to try it if I can figure it out.There is no way I can justify buying another card until I am further along, although I would like to.I have to wait until I have tested the waters a bit more and been successful with OpenCL based HW. Trying to work around the lack of plug and play is a good way to learn more.I am very conservative when it comes to spending these days.Back to photogrammetry/SLAM/3D reconstruction.If any of you can help me by suggesting either OpenCL or OpenGL /shader based workflows I would be very grateful.With my AMD card, I am hoping that the code will also be open, no proprietary vendor lock in. Thats very important now when we consider that this technology is poised to take so many jobs once it matures a bit. There needs to be accountability on at least some level.Look at how evil it is for Youtube videos to be arbitrarily demonetized with no info as to why. Thats what we're going to be dealing with on a massive scale as people are replaced by AI and its binary blobs.AI making arbitrary decisions is like carte blanche to screw anybody and everybody - as Dave says "without even a by or leave".

author: lukier
author_rank: Unknown
timestamp: October 22, 2018, 07:22:37 pm
content: Quote from: cdev on October 22, 2018, 06:47:55 pmSorry, I played a bit with OpenCL back in 2012 and as much as I hate Nvidia's vendor lock in I hate OpenCL even more.The reason why most research uses CUDA and pretty much entire deep learning is because OpenCL sucks big time.OpenCL is still not yet at the level CUDA was 10 years ago (with CUDA 2.0 let's say). OpenCL 2.1 that supports more or less normal C++ is still partially rolled out and the same with SYCL (that allows more seamless mixing of host and device code). CUDA was there ages ago!For example, with CUDA back in 2013 I was able to run fixed size dense matrix algebra using Eigen inside the kernels, with vectorization, then some Lie group algebra that's based on Eigen and then a bit later even use automatic differentiation to compute partial jacobians on the GPU, get the results back to the CPU to do Levenberg-Marquardt steps.At that time OpenCL was only C99 with pretty much no functions, every matrix multiplication had to be written by hand and for defined types (no templates). Don't get me even started about these moronic disjoint address spaces that actually changed the type system, even if you wanted to reuse some code (with non-portable AMD C++ extension, kind of defying the point of OpenCL) it broke everything. I ran away from OpenCL as quickly as I can.Also, I've heard that the performance-wise it is not that great, as the idea was to have the same code across the CPU, GPU and other compute platforms (DSP?FPGA?), but in practice (not surprising) one has to write and tune kernels to the type of compute device (and it's operating principles) that will execute these kernels.

author: cdev
author_rank: Super Contributor
timestamp: October 22, 2018, 09:38:12 pm
content: Thank you for this much more specific information about what kinds of math are more difficult for you in OpenCL. My current setup supports version 2.1 according to the ViennaCL benchmarks but some programs just recognize it as 1.2 because I have cobbled together a sort of Frankenstein system because I don't want to have to switch to Ubuntu just to run OpenCL apps. But, I have it working, so I'm happy. The usual acceleration when an app supports OpenCL is huge, 10 times is fairly typical. I'm sure that is the case for Cuda too. Note that my current card cost me less than $200.Do you know of any cross-platform GPU computing aware benchmarks that can be used to compare the two platforms in a hardware agnostic way?

author: janoc
author_rank: Super Contributor
timestamp: October 22, 2018, 09:44:00 pm
content: Pretty much as lukier said - people don't use CUDA because they love it so much or Nvidia is paying them but because it is the only viable game in town.If you don't like it, feel free to go old school and rewrite the calculations using e.g. compute shaders. That works too and the performance should be similar to CUDA (it is basically the same thing) but you will quickly wish you had that CUDA to do the boilerplate and housekeeping necessary for you.SLAM, photogrammetry, deep learning - none of these can be done realistically for actually useful sized tasks unless you are willing to spend money, whether on hardware or something like Google or Amazon cloud compute services. CPU-only variants are good for toy-sized problems only, it would take ages to process any real world dataset like that.So if you can't afford a decent GPU, don't waste your money buying an underpowered one. You will be fighting an uphill battle and will give up in disgust sooner than later. Many of these things need iterations where you tweak some parameters, run the code for a few hours, then tweak some more, run it again, until you get something acceptable. If those few hours per round turn into days you will quickly give up.And regardless of that - using that measuring tape is still going to be faster than any of this.E.g. just today I was playing at work with a deep learning based animation system presented at this year's SIGGRAPH. It can generate data for steering a physically controlled (PID and such) simulated robot by learning from an example animation, such as a motion captured keyframe animation performed by a human actor. Complex problem but once trained it runs in real time. However, the training on a single beefy computer with 16 threads in parallel using a single GPU for one animation takesover a day. Without GPU it is completely nonviable, it would take weeks.

author: cdev
author_rank: Super Contributor
timestamp: October 23, 2018, 01:02:54 am
content: Could you take a look at the ROCm "HIP" and related work, (AMD open source cross-compiler allowing work with Cuda code on AMD HW) effort and let me know what you think of the viability of their approach?Just a quick look.https://github.com/ROCm-Developer-Tools/HIPalsohttps://gpuopen.comandhttps://rocm.github.io/My current HW (older 6 core AMD "bulldozer" board lacking PCIe 3 atomics support) doesn't support it now.. My GPU which is a Polaris/Ellsmere type would now if I had a newer (Ryzen/Epyx or newer Intel-based) MB and CPU but its my understanding that future revisions of their code 'may' (or may not)I get the impression it may even work now but if so I am on my own and its totally unsupported.I cant afford to damage anything so I am waiting to try it until they say they can get it running properly, or I will just buy another GPU and MB once I know I need it and have outgrown everything I have now (I'm not even close, I am just feeling the waters now and don't know practically anything).I just want to know your opinion on the viability of doing what they are doing and running Cuda code on it. Note that Ive used several different similar programs to run simple Cuda programs and its worked for some but not all of them. gpuocelot - two years ago I had it running and it worked, and I successfully ran cuda test programs on it. I forget what hung me up, something did though. It wasn't a complete tool. And now its a dead project without any support.None of the similar tools quite work now, none are viable for anything complex though, and gpuocelot is really old and broken- it no longer compiles for me now.

author: janoc
author_rank: Super Contributor
timestamp: October 24, 2018, 03:52:31 pm
content: Quote from: cdev on October 23, 2018, 01:02:54 amI don't know whether that is viable. Crosscompiling CUDA code to C++ won't make it magically work on a non-CUDA GPUs - it still needs drivers and other infrastructure which isn't in place. AMD is very much non-existent in the high performance/research computing area. Given that no major project uses this it is likely a proof of concept at best and not production-ready tool.I don't really understand what are you trying to do here. You won't save any money trying to push this stuff to CPUs or low end cheap GPUs. There is no magic in there - if the hardware is underpowered, it makes zero difference what framework or tool you use to manage the calculations.Your current HW is hopelessly outdated, Bulldozer was released 7 years ago so no wonder the code doesn't work with it.Also, do you want to rewrite the major computer vision frameworks needed for running SLAM or any photogrammetry (such as OpenCV) to use this yourself? Because otherwise I fail to see how it even could be relevant for you. I am pretty certain you don't have the scientific knowledge necessary to pull it off.

author: cdev
author_rank: Super Contributor
timestamp: October 24, 2018, 08:39:45 pm
content: You're right, I don't have the skill to "rewrite the major CV frameworks".By the way, maybe I didn't make it clear, Ive sort of diverged here to a more general discussion of GPU computing platforms- sorry if I didnt make that clear.I am wondering if the situation with AMD is really as dire as you make it out to be.It may well be, but maybe not.But my AMD card is fairly current. So I am hoping I dont need to replace it that soon.Also, for example, OpenCV supports OpenCL as do forks of a great many other frameworks.It may not be plug and play (Thats why I was asking about the ROCm tools) but its out there.There is another issue too, proprietary code shouldn't be put in a position where binary blobs make decisions, perhaps life or death decisions - soon - that are totally opaque to human examination and criticism.

author: janoc
author_rank: Super Contributor
timestamp: October 25, 2018, 10:02:30 pm
content: Quote from: cdev on October 24, 2018, 08:39:45 pmYes and as you were told already, OpenCL on your old hardware is totally hopeless performance-wise. And that RaspberryPi was hopefully a joke, you don't want to run mapping (not navigation - that could work) SLAM on that, do you? You obviously have no idea how much computing power the job you want to do actually requires.Quote from: cdev on October 24, 2018, 08:39:45 pmAre you planning to build an armed drone or what? Otherwise I fail to see the relevance of this to your problem.Also, open source is great - but when the open source solution either doesn't exist or is unsupported/doesn't work/is pain to use, then you either go with the proprietary oneand solve your problemor keep talking about how morals - but the problem will be left unsolved. The third option is to actually fix the open source tools but most people don't have neither the means nor time for that.I would also rather use open source drivers and libraries but when the state of the art is proprietary and the open solutions are not really comparable, then there is not much left to debate, IMO.

author: cdev
author_rank: Super Contributor
timestamp: October 26, 2018, 12:27:34 am
content: This is a good essay on the pros of the OpenCL like approach.https://mathema.tician.de/the-state-of-opencl-for-scientific-computing-in-2018/I have a semi-current video card, as I said. On a very few applications it is actually faster than their more recent Vega consumer cards. Its driver is more mature and at the current time it actually better supported than their Vega cards, by a great many applications (although that is changing)Its performance on benchmarks is around 65-75% of what I would get paying twice or three times as much for AMDs newest consumer HW. And it cost around a third as much. Thats value, its not "obsolete". That I plan to keep. In a year or so, once I get my feet wet with what Ive got, I definitely plan to upgrade the CPU, but I would prefer to wait as long as I can, in particular until Ive reached the limits of my current HW, which I definitely have not done yet.No, I am not planning to build anything like that, the thing I want to do is learn how to build and train models to solve everyday problems using neural nets. Not because that is guaranteed to be the best approach, just because I want to see how it works and how it differs from the more conventional approaches in coding.

author: janoc
author_rank: Super Contributor
timestamp: October 26, 2018, 08:03:43 pm
content: Quote from: cdev on October 26, 2018, 12:27:34 amDidn't you say that you want to do photogrammetry/SLAM to scan your room? Neither of that has nothing whatsoever to do with neural nets and machine learning.I feel you don't really understand what these terms actually mean and what the software is really doing.If you want to learn how SLAM works, this course is a very good start to give you an introduction into the basics. There are newer techniques, such as dense SLAM, but they all start from the same foundation:http://www.joansola.eu/JoanSola/eng/course.htmlPhotogrammetry basics are quite well explained here:https://www.cse.wustl.edu/~furukawa/papers/fnt_mvs.pdfThe problems are related because SLAM uses photogrammetric techniques to create the map and most photogrammetry software needs to calculate where the camera was when every picture was taken to be able to reconstruct the scene.Again, none of this has anything to do with neural networks or machine learning. There are some never works that use machine learning to improve some aspects of the above but neither SLAM nor photogrammetry are machine learning problems.If you want to do machine learning and don't have the hardware necessary for it (training actually useful models is frequently beyond the means of a single PC even with a modern GPU), you will do better if you pay for computing time e.g. on Amazon or elsewhere - it is not too expensive, most machine learning frameworks support this and you get to train even large models that would take you weeks at home otherwise. But once again, you don't need any of this for scanning/reconstructing your place.