Brian Bi
Exercise 6.8.1 Yes. $$I * A = I A I^t = A$$ and $$(PQ) * A = (PQ)A(PQ)^t = PQAQ^tP^t = P(QAQ^t)P^t = P * (Q * A)$$.
Exercise 6.8.2 If $$g$$ is in the stabilizer of some subset $$S \subseteq G$$, then it's not hard to see that $$aga^{-1}$$ is in the stabilizer of $$aS$$. Conversely, every element $$g'$$ that stabilizes $$aS$$ must have this form, since $$a^{-1}g'a$$ would stabilize $$S$$, implying that $$g' = aga^{-1}$$ where $$g$$ stabilizes $$S$$. By Proposition 6.8.1, the stabilizer of the coset $$aH$$ is therefore the conjugate subgroup $$aHa^{-1}$$.
Exercise 6.8.3 Let $$s$$ be some vertex in $$S$$. The stabilizer of $$s$$ consists of the identity operation and the reflection across the diagonal containing $$s$$. Call this reflection $$r$$, so $$H = \{1, r\}$$. The orbit is all of $$S$$. There are $$8 \div 2 = 4$$ left cosets $$aH$$, namely $$\{1, r\}, \{\rho, \rho r\}, \{\rho^2, \rho^2 r\}, \{\rho^3, \rho^3 r\}$$, which are respectively mapped by $$\epsilon$$ to the vertices $$1s = rs = s, \rho s = \rho r s, \rho^2 s = \rho^2 r s, \rho^3 s = \rho^3 r s$$, thus hitting all four vertices exactly once. That is, $$\epsilon$$ is bijective as expected.
Exercise 6.8.4 The subgroup $$H$$ consists of all $$(n-1)!$$ permutations on the indices $$\{2, 3, \ldots, n\}$$. Let $$a \in S_n$$. Then the coset $$aH$$ consists of permutations $$\sigma$$ such that $$\sigma(1) = a(1)$$, since $$ah(1) = a(1)$$ for all $$h \in H$$, since $$h(1) = 1$$. Since the coset $$aH$$ contains $$(n-1)!$$ elements and there are $$(n-1)!$$ permutations with $$\sigma(1) = a(1)$$, the coset $$aH$$ must contain all of them. The map (6.8.4) sends the coset $$aH$$ to $$a(1)$$, by definition.