Yes, you can! If you buy fresh lemongrass stalks from the grocery store, ensure they have a portion of the base intact. Place them in a jar with an inch or two of water. In a few weeks, they should develop roots, and you can plant them in soil or coconut coir.