  General Relativity predicts only two tensor polarization modes for
gravitational waves while at most six possible polarization modes of
gravitational waves are allowed in the general metric theory of gravity. The
number of polarization modes is totally determined by the specific modified
theory of gravity. Therefore, the determination of polarization modes can be
used to test gravitational theory. We introduce a concrete data analysis
pipeline for a single space-based detector such as LISA to detect the
polarization modes of gravitational waves. Apart from being able to detect
mixtures of tensor and extra polarization modes, this method also has the added
advantage that no waveform model is needed and monochromatic gravitational
waves emitted from any compact binary system with known sky position and
frequency can be used. We apply the data analysis pipeline to the reference
source J0806.3+1527 of TianQin with 90-days' simulation data, and we show that
$\alpha$ viewed as an indicative of the intrinsic strength of the extra
polarization component relative to the tensor modes can be limited below 0.5
for LISA and below 0.2 for Taiji. We investigate the possibility to detect the
nontensorial polarization modes with the combined network of LISA, TianQin and
Taiji and find that $\alpha$ can be limited below 0.2.
