The curves $y=x^2$ and $y=8/x$ meet where $x^2=8/x\Rightarrow x=2$ (and $y=4$). The region in $x\ge 0$ with $1\le y\le x^2$ and $xy\le 8$ splits at $x=2$:

For $1\le x\le 2$: bounded above by $y=x^2$ (below the hyperbola too) and below by $y=1$.

For $2\le x\le 8$: bounded above by $y=8/x$ (since $x^2$ now exceeds $8/x$ and the hyperbola constraint binds) and below by $y=1$, ending at $x=8$ where $y=1$.

$A={\int }_1^2(x^2-1)dx+{\int }_2^8\left(\frac{8}{x}-1\right)dx$

$={\left[\frac{x^3}{3}-x\right]}_1^2+[8\ln x-x{]}_2^8$

$=\left(\frac{8}{3}-2-\frac{1}{3}+1\right)+(8\ln 8-8-8\ln 2+2)$

$=\frac{4}{3}+8\ln 4-6=8\ln 4+\frac{4}{3}-6=16\ln 2-\frac{14}{3}$

$=\frac{48\ln 2-14}{3}=\frac{2}{3}(24\ln 2-7)$.

Option (3).