kkaefer/mapnik — comment on commit 970dad548f48ebb0dc6214a4b25fa62395bebcc0
File:  line 0
By kunitoki:

wonderful ! i've started the same task here, and started from boost::threadpool too :) btw we should merge the efforts here.

from my tests ( https://github.com/mapnik/mapnik/issues/849#issuecomment-2518513 ), most of the time is spent rendering, and there is no need in waiting that all the layers finish fetching all the data, but if layer 1 finished fetching but layer 2 is still working, then layer 1 can start rendering. obviously the opposite, if layer 1 is still fetching but layer 2 has finished and wants to draw, layer 2 will wait until layer 1 finished drawing. pretty easy with boost::thread_group too.

here is some ideas:

//========================================================================================
/**
Compile with:

g++ -pthread -lboost_thread-mt -o test main.cpp
*/

#include
#include
#include
#include

#include
#include
#include

static boost::mutex io_mutex;
static void write(std::string s)
{
boost::mutex::scoped_lock sl(io_mutex);
std::cout << s;
}

typedef boost::shared_ptr mutex_ptr;

class task
{
public:
task()
{
mutex_ = boost::make_shared();
}

mutex_ptr mutex()
{
return mutex_;
}

virtual void run() = 0;

protected:

mutex_ptr mutex_;
};

typedef boost::shared_ptr task_ptr;

class task_group
{
public:
task_group()
{
}

void add_task(task_ptr t)
{
group_.create_thread(boost::bind(&task::run, t));
}

void join_all()
{
group_.join_all();
}

protected:

boost::thread_group group_;
};

class chained_task : public task
{
public:
chained_task(task_ptr prev, int number)
: prev_(prev),
number_(number)
{
}

protected:
void run()
{
{
boost::lock_guard ourlock(*mutex());

std::stringstream s1;
boost::this_thread::sleep(boost::posix_time::seconds(number_));
s1 << "chained_task(" << number_ << ") executing overlappable code... done !" << std::endl;
write(s1.str());

if (prev_)
{
std::stringstream s2;
boost::lock_guard prevlock(*prev_->mutex());
s2 << "chained_task(" << number_ << ") waiting previous to finish... done !" << std::endl;
write(s2.str());
}

std::stringstream s3;
boost::this_thread::sleep(boost::posix_time::seconds(number_));
s3 << "chained_task(" << number_ << ") executing code... done !" << std::endl;
write(s3.str());
}
}

task_ptr prev_;
int number_;
};

typedef boost::shared_ptr chained_task_ptr;

int main (int argc, char* argv[])
{
task_group g;

chained_task_ptr t1 = boost::make_shared(task_ptr(), 1);
chained_task_ptr t2 = boost::make_shared(t1, 3);
chained_task_ptr t3 = boost::make_shared(t2, 5);

g.add_task(t1);
g.add_task(t2);
g.add_task(t3);

g.join_all();

return 0;
}

related to this... we should give a bit more testing on all the datasources, as i've encountered some problems with multi threaded fetching of features from a sqlite database, which should be opened with the FULLMUTEX or with serialized mode enabled.