Draw the FBD of both the blocks. In the reference frame of block M, the block m will experience pseudo force. Apply Newton’s second law on the block m along the inclined plane and Newton’s first law along the perpendicular to the inclined plane. For block M apply Newton’s second law along the horizontal.
Let the acceleration of the prism be in the backward direction. Consider the motion of the smaller block from the frame of the prism. The forces on the block are
(i) N normal force,
(ii) mg downward (gravity), and
(iii) m a0 forward(pseudo).

The block slides down the plane. Components of the forces parallel to the incline give
ma0cosθ + mgsinθ = ma
or a = a0cosθ + gsinθ ... (i)
Components of the force perpendicular to the incline give
N + ma0sinθ = mgcosθ . ...(ii)
Now, consider the motion of the prism from the lab frame. No pseudo force is needed as the frame used is inertial. The forces acting now are
(i) Mg downward,
(ii) N normal to the incline (by the block), and
(iii) N′ upward (by the horizontal surface).
Horizontal components give
