Suppose a, b, ∈ R and a, b≠1. If the system of equation ax + y + z = 0, x + by + z = 0 , x + y + 2z = 0
has a non-trivial solution, then
a + b = 2
a + b = ab
a+1b=2
a + b = 0
From (1) and (3)
(a – 1)x – z = 0
and from (2) and (3)
(b-1) y=z
∴ x1(1-a)=y1(1-b)=z1
Putting in (1) we get
a1-a+11-b+1=0
⇒ a1-a=-11-b
⇒ 1-b=-1+a
⇒ a+b=2