Write $\mathcal{F} = \colim \mathcal{F}[n]$ where the colimit is over the multiplicative system of integers invertible in $K$. Since cohomology commutes with filtered colimits in our situation (for a precise reference see Lemma \ref{lemma-base-change-Rf-star-colim}), it suffices to prove the lemma for $\mathcal{F}[n]$. Thus we may assume $\mathcal{F}$ is a sheaf of $\mathbf{Z}/n\mathbf{Z}$-modules for some $n$ invertible in $K$ (we will use this at the very end of the proof). In the proof we use the short hand $X \times_K Y$ for the fibre product over $\Spec(K)$. We will prove the lemma by induction on $\dim(X) + \dim(Y)$. The lemma is trivial if $\dim(X) \leq 0$, since in this case $U$ is an open and closed subscheme of $X$. Choose a point $z \in X \times_K Y$. We will show the stalk at $\overline{z}$ is an isomorphism. \medskip\noindent Suppose that $z \mapsto x \in X$ and assume $\text{trdeg}_K(\kappa(x)) > 0$. Set $X' = \Spec(\mathcal{O}_{X, x}^{sh})$ and denote $U' \subset X'$ the inverse image of $U$. Consider the base change $$ \xymatrix{ Y \times_K X' \ar[d]_{q'}  & Y \times_K U' \ar[l]^{h'} \ar[d]^{p'} \\ X' & U' \ar[l]_{j'} } $$ of our diagram by $X' \to X$. Observe that $X' \to X$ is a filtered colimit of \'etale morphisms. By smooth base change in the form of Lemma \ref{lemma-smooth-base-change-general} the pullback of $q^{-1}Rj_*\mathcal{F} \to Rh_*p^{-1}\mathcal{F}$ to $X'$ to $Y \times_K X'$ is the map $(q')^{-1}Rj'_*\mathcal{F}' \to Rj'_*(p')^{-1}\mathcal{F}'$ where $\mathcal{F}'$ is the pullback of $\mathcal{F}$ to $U'$. (In this step it would suffice to use \'etale base change which is an essentially trivial result.) So it suffices to show that $(q')^{-1}Rj'_*\mathcal{F}' \to Rj'_*(p')^{-1}\mathcal{F}'$ is an isomorphism in order to prove that our original map is an isomorphism on stalks at $\overline{z}$. By Lemma \ref{lemma-strictly-henselian} there is a separably closed field $L/K$ such that $X' = \lim X_i$ with $X_i$ affine of finite type over $L$ and $\dim(X_i) < \dim(X)$. For $i$ large enough there exists an open $U_i \subset X_i$ restricting to $U'$ in $X'$. We may apply the induction hypothesis to the diagram $$ \vcenter{ \xymatrix{ Y \times_K X_i \ar[d]_{q_i}  & Y \times_K U_i \ar[l]^{h_i} \ar[d]^{p_i} \\ X_i & U_i \ar[l]_{j_i} } } \quad\text{equal to}\quad \vcenter{ \xymatrix{ Y_L \times_L X_i \ar[d]_{q_i}  & Y_L \times_L U_i \ar[l]^{h_i} \ar[d]^{p_i} \\ X_i & U_i \ar[l]_{j_i} } } $$ over the field $L$ and the pullback of $\mathcal{F}$ to these diagrams. By Lemma \ref{lemma-base-change-Rf-star-colim} we conclude that the map $(q')^{-1}Rj'_*\mathcal{F}' \to Rj'_*(p')^{-1}\mathcal{F}$ is an isomorphism. \medskip\noindent Suppose that $z \mapsto y \in Y$ and assume $\text{trdeg}_K(\kappa(y)) > 0$. Let $Y' = \Spec(\mathcal{O}_{X, x}^{sh})$. By Lemma \ref{lemma-strictly-henselian} there is a separably closed field $L/K$ such that $Y' = \lim Y_i$ with $Y_i$ affine of finite type over $L$ and $\dim(Y_i) < \dim(Y)$. In particular $Y'$ is a scheme over $L$. Denote with a subscript $L$ the base change from schemes over $K$ to schemes over $L$. Consider the commutative diagrams $$ \vcenter{ \xymatrix{ Y' \times_K X \ar[d]_f  & Y' \times_K U \ar[l]^{h'} \ar[d]^{f'} \\ Y \times_K X \ar[d]_q  & Y \times_K U \ar[l]^h \ar[d]^p \\ X & U \ar[l]_j } } \quad\text{and}\quad \vcenter{ \xymatrix{ Y' \times_L X_L \ar[d]_{q'}  & Y' \times_L U_L \ar[l]^{h'} \ar[d]^{p'} \\ X_L \ar[d] & U_L \ar[l]^{j_L} \ar[d] \\ X & U \ar[l]_j } } $$ and observe the top and bottom rows are the same on the left and the right. By smooth base change we see that $f^{-1}Rh_*p^{-1}\mathcal{F} = Rh'_*(f')^{-1}p^{-1}\mathcal{F}$ (similarly to the previous paragraph). By smooth base change for $\Spec(L) \to \Spec(K)$ (Lemma \ref{lemma-base-change-field-extension}) we see that $Rj_{L, *}\mathcal{F}_L$ is the pullback of $Rj_*\mathcal{F}$ to $X_L$. Combining these two observations, we conclude that it suffices to prove the base change map for the upper square in the diagram on the right is an isomorphism in order to prove that our original map is an isomorphism on stalks at $\overline{z}$\footnote{Here we use that a ``vertical composition'' of base change maps is a base change map as explained in Cohomology on Sites, Remark \ref{sites-cohomology-remark-compose-base-change}.}. Then using that $Y' = \lim Y_i$ and argueing exactly as in the previous paragraph we see that the induction hypothesis forces our map over $Y' \times_K X$ to be an isomorphism. \medskip\noindent Thus any counter example with $\dim(X) + \dim(Y)$ minimal would only have nonisomorphisms $q^{-1}Rj_*\mathcal{F} \to Rh_*p^{-1}\mathcal{F}$ on stalks at closed points of $X \times_K Y$ (because a point $z$ of $X \times_K Y$ is a closed point if and only if both the image of $z$ in $X$ and in $Y$ are closed). Since it is enough to prove the isomorphism locally, we may assume $X$ and $Y$ are affine. However, then we can choose an open dense immersion $Y \to Y'$ with $Y'$ projective. (Choose a closed immersion $Y \to \mathbf{A}^n_K$ and let $Y'$ be the scheme theoretic closure of $Y$ in $\mathbf{P}^n_K$.) Then $\dim(Y') = \dim(Y)$ and hence we get a ``minimal'' counter example with $Y$ projective over $K$. In the next paragraph we show that this can't happen. \medskip\noindent Consider a diagram as in the statement of the lemma such that $q^{-1}Rj_*\mathcal{F} \to Rh_*p^{-1}\mathcal{F}$ is an isomorphism at all non-closed points of $X \times_K Y$ and such that $Y$ is projective. The restriction of the map to $(X \times_K Y)_{K^{sep}}$ is the corresponding map for the diagram of the lemma base changed to $K^{sep}$. Thus we may and do assume $K$ is separably algebraically closed. Choose a distinguished triangle $$ q^{-1}Rj_*\mathcal{F} \to Rh_*p^{-1}\mathcal{F} \to Q \to (q^{-1}Rj_*\mathcal{F})[1] $$ in $D((X \times_K Y)_\etale)$. Since $Q$ is supported in closed points we see that it suffices to prove $H^i(X \times_K Y, Q) = 0$ for all $i$, see Lemma \ref{lemma-vanishing-closed-points}. Thus it suffices to prove that $q^{-1}Rj_*\mathcal{F} \to Rh_*p^{-1}\mathcal{F}$ induces an isomorphism on cohomology. Recall that $\mathcal{F}$ is annihilated by $n$ invertible in $K$. By the K\"unneth formula of Lemma \ref{lemma-kunneth-one-proper} we have \begin{align*} R\Gamma(X \times_K Y, q^{-1}Rj_*\mathcal{F}) &= R\Gamma(X, Rj_*\mathcal{F}) \otimes_{\mathbf{Z}/n\mathbf{Z}}^\mathbf{L} R\Gamma(Y, \mathbf{Z}/n\mathbf{Z}) \\ & = R\Gamma(U, \mathcal{F}) \otimes_{\mathbf{Z}/n\mathbf{Z}}^\mathbf{L} R\Gamma(Y, \mathbf{Z}/n\mathbf{Z}) \end{align*} and $$ R\Gamma(X \times_K Y, Rh_*p^{-1}\mathcal{F}) = R\Gamma(U \times_K Y, p^{-1}\mathcal{F}) = R\Gamma(U, \mathcal{F}) \otimes_{\mathbf{Z}/n\mathbf{Z}}^\mathbf{L} R\Gamma(Y, \mathbf{Z}/n\mathbf{Z}) $$ This finishes the proof.