Question: I made a related post to [Stack Overflow][1] the other day, but I figured that people here might e.g. know some bioinfo tools that do what I want..

So my data is in this kind of format (tab separated parsed from a sam file):

k_12_exc_170	272	FR667691.1	2646	ACATATAAGGTT
k_12_exc_132	256	FR667691.1	8100	AACATGGCGAGA
k_14_exc_1547	16	FR667691.1	8669	ATTATAAAGAATTT
k_13_exc_6130	272	FR667691.1	10710	GAGAAAGAACCAT
k_13_exc_6130	272	FR667691.1	10729	GAGAAAGAACCAT
k_13_exc_8289	16	FR667691.1	10827	TATTTATTGATGG
k_14_exc_19763	272	FR667691.1	15800	TTTAAAATGATTAC
k_15_exc_9848	16	FR667691.1	15800	TTTAAAATGATTACT
k_13_exc_3692	16	FR667691.1	16119	TGGACGAATAAGT
k_14_exc_2426	272	FR667691.1	16119	TGGACGAATAAGTT
k_12_exc_3570	272	FR667691.1	16436	GAACCATTCGAG

What I want to do, is to print all lines as pairs where the value in column 4 of the (to be) "right pair" is at least 100 larger than value in column 4 of the "left pair", but at the same time at most 200 larger. In other words, these lines define chromosomal regions, and I want to output each possible case where two lines are within the rules stated above. I can start this task from the sam file too. These files have ca. 100 k lines each and there can be thousands of them. They are sorted based on columns 3 and 4. Currently I'm doing pretty much what was suggested at Stack Overflow, except I first split each file into 1000 row files with additional 100 tail rows from the previously split file in head (so that many pairs would not be lost along the way). While this is considerably faster than whatever I was doing before, I can't help thinking that there should be some simpler and faster way to do it.. perhaps with bedtools or samtools or something like that?

With example data, only such row should be printed (but I can deal with other formats too but this info should be available):

k_13_exc_6130	272	FR667691.1	10710	GAGAAAGAACCAT k_13_exc_8289	16	FR667691.1	10827	TATTTATTGATGG

So in essence, I want to do conditional join..

[1]: http://stackoverflow.com/questions/36535289/print-all-rows-that-are-in-specified-value-range-based-on-specific-column-as-pai

Answer: Here is a quick solution in C. Not beautiful but it should give you an idea of how an optimal solution might look like.

#include
#include

const long LOWER = 100;
const long UPPER = 200;
const long MAX_LINES = 1000000;

struct line {
char *foo, *bar, *	baz;
long derp, herp;
};

int main()
{
struct line *lines = malloc(MAX_LINES * sizeof(*lines));
struct line *fillptr, *left, *right_upper, *right_lower;
fillptr = left = right_upper = right_lower = lines;

while ( 5 == scanf("%ms %li %ms %li %ms\n", &fillptr->foo, &fillptr->derp, &fillptr->bar, &fillptr->herp, &fillptr->baz)) {
fillptr++;
}

// assumes lines are sorted by `herp`.
while ( left < fillptr) {
while (right_lower->herp < left->herp + LOWER && right_lower < fillptr) {
right_lower++;
}
while (right_upper->herp < left->herp + UPPER && right_upper < fillptr) {
right_upper++;
}
struct line *ptr = right_lower;
while (ptr < right_upper) {
printf("%s\t%li\t%s\t%li\t%s", left->foo, left->derp, left->bar, left->herp, left->baz);
printf("%s\t%li\t%s\t%li\t%s\n", ptr->foo, ptr->derp, ptr->bar, ptr->herp, ptr->baz);
ptr++;
}
free(left->foo);
free(left->bar);
free(left->baz);
left++;
}

free(lines);
return 0;
}

Untested except for that tiny example. ☺