Angle between a and b is θ = 120° and |a| = |b| = 2
Now, [(a + 3b) x (3a + b)]2 = [a x 3a + a x b + 9(b x a) + 3b x b]2
= [a x b - 9(a x b)]2 (\(\because\) a x a = 0, b x b = 0 and b x a = -(a x b))
= [-8(a x b)]2 = 64(a x b)2
= 64 (|a|.|b| sin θ \(\hat n\))2 (\(\because\) a x b = |a|.|b| sin θ \(\hat n\) )
= 64 |a|2.|b|2 sin2θ (\(\hat n^2=\hat n.\hat n=1\))
= 64 x 22 x 22 x sin2120° (\(\because\) |a| = |b| = 2)
= 64 x 16 x 3/4 = 64 x 12 = 768 (\(\because\) sin120° = sin 60° = \(\frac{\sqrt3}2\))