A jigsaw isn’t the best tool for reliably cutting straight lines, but that doesn’t mean it can’t be done. With a little preparation, you should be able to do it relatively easily.
Choose a wider blade for cutting straight lines, as its rigidity will make your task much easier. Narrower blades are more flexible, which makes them ideal for cutting curves but also means straight cuts will be very difficult.
You can use another block of wood, a ruler, or a straightedge to guide the jigsaw’s shoe in a straight line.