In this talk, I will first give a brief introduction to some general techniques on the design of fast solvers for large scale algebraic systems. Using model problems such as fluid-structure interaction (FSI) and magneto-hydrodymics (MHD), I will demonstrate how optimal solvers can be designed and analyzed for coupled PDE systems by using the underlying mathematical and physical features and properly designed structure-preserving finite element discretization schemes.