Hint: Use Gauss' law.

(a) Step 1: Find the electric field for r<R.

Let us consider a sphere S of radius R and two hypothetic spheres of radius r<R and r>R.

Now, for point r<R, electric field intensity will be given by,

$\oint \mathrm{E}.\mathrm{dS}=\frac{1}{{\mathrm{\epsilon }}_0}\int \mathrm{\rho dV} [\mathrm{For} \mathrm{dV}, \mathrm{V}=\frac{4}{3}{\mathrm{\pi r}}^3\Rightarrow \mathrm{dV}=3\times \frac{4}{3}{\mathrm{\pi r}}^2\mathrm{dr}=4{\mathrm{\pi r}}^2\mathrm{dr}]$
$\Rightarrow \oint \mathrm{E}.\mathrm{dS}=\frac{1}{{\mathrm{\epsilon }}_0}4\mathrm{\pi K}{\int }_0^{\mathrm{r}}{\mathrm{r}}^3\mathrm{dr} (\because \mathrm{\rho }(\mathrm{r})=\mathrm{Kr})$
$\Rightarrow \left(\mathrm{E}\right) 4{\mathrm{\pi r}}^2=\frac{4\mathrm{\pi K}}{{\mathrm{\epsilon }}_0}\frac{{\mathrm{r}}^4}{4}$
$\Rightarrow \mathrm{E}=\frac{1}{4{\mathrm{\epsilon }}_0}{\mathrm{Kr}}^2$

Here, the charge density is positive.

So, the direction of E is radially outwards.

Step 2: Find the electric field for r>R.

For points r>H, electric field intensity will be given by,

$\oint \mathrm{E}.\mathrm{dS}=\frac{1}{{\mathrm{\epsilon }}_0}\int \mathrm{\rho dV}$
$\Rightarrow \mathrm{E}\left(4{\mathrm{\pi r}}^2\right)=\frac{4\mathrm{\pi K}}{{\mathrm{\epsilon }}_0}{\int }_0^{\mathrm{R}}{\mathrm{r}}^3\mathrm{dr}=\frac{4\mathrm{\pi K}}{{\mathrm{\epsilon }}_0}\frac{{\mathrm{R}}^4}{4}$
$\Rightarrow \mathrm{E}=\frac{\mathrm{K}}{4{\mathrm{\epsilon }}_0}\frac{{\mathrm{R}}^4}{{\mathrm{r}}^2}$

Th charge density is again positive. So, the direction of E is radially outward.

(b) Step 3: Identify the location of the protons.

The two protons must be on the opposite sides of the centre along a diameter following the rule of symmetry. This can be shown by the figure given below. Charge on the sphere,

Step 4: Find the net force on the protons.

$\mathrm{q}={\int }_0^{\mathrm{R}}\mathrm{\rho dV}={\int }_0^{\mathrm{R}}\left(\mathrm{Kr}\right)4{\mathrm{\pi r}}^2\mathrm{dr}$
$\mathrm{q}=4\mathrm{\pi K}\frac{{\mathrm{R}}^4}{4}=2\mathrm{e}$
$\therefore \mathrm{K}=\frac{2\mathrm{e}}{{\mathrm{\pi R}}^4}$

If protons 1 and 2 are embedded at distance r from the centre of the sphere as shown, then attractive force on proton 1 due to charge distribution is:

${\mathrm{F}}_1=\mathrm{eE}=\frac{-{\mathrm{eKr}}^2}{4{\mathrm{\epsilon }}_0}$

Repulsive force on proton 1 due to proton 2 is:

${\mathrm{F}}_2=\frac{{\mathrm{e}}^2}{4{\mathrm{\pi \epsilon }}_0{\left(2\mathrm{r}\right)}^2}$

The net force on proton 1,

$\mathrm{F}={\mathrm{F}}_1+{\mathrm{F}}_2$
$\mathrm{F}=\frac{-{\mathrm{eKr}}^2}{4{\mathrm{\epsilon }}_0}+\frac{{\mathrm{e}}^2}{16{\mathrm{\pi \epsilon }}_0{\mathrm{r}}^2}$

So, $\mathrm{F}=[\frac{-{\mathrm{er}}^2}{4{\mathrm{\epsilon }}_0}\frac{\mathrm{Ze}}{{\mathrm{\pi R}}^4}+\frac{{\mathrm{e}}^2}{16{\mathrm{\pi \epsilon }}_0{\mathrm{r}}^4}]$

Step 5: Find the distance between the protons.

Thus, the net force on proton 1 will be zero, when:

$\frac{{\mathrm{er}}^{2}2\mathrm{e}}{4{\mathrm{\epsilon }}_0{\mathrm{\pi R}}^4}=\frac{{\mathrm{e}}^2}{16{\mathrm{\pi \epsilon }}_0\mathrm{r}}$
$\Rightarrow {\mathrm{r}}^4=\frac{{\mathrm{R}}^4}{8}$
$\Rightarrow \mathrm{r}=\frac{\mathrm{R}}{{\left(8\right)}^{1/4}}$

This is the distance of each of the two protons from the centre of the sphere.