The formation of grain boundary (GB) by non-homogeneous plastic deformation was captured directly through transmission electron microscopy in situ compression test for body centered cubic iron. Stress inhomogeneity was purposely induced by a concaved grain shape using a focused ion beam, leading to enhanced interaction of dislocations by the activation of various slip systems. Wall-like tangled dislocations were gathered around the boundary at the apex or the bottom of the concaved surface with various stress/strain fields in an early deformation stage. The wall-like tangled dislocations further evolved into sharp grain boundaries with increasing plastic strain, analogous to the array of geometrically necessary dislocations. In contrast, the adjacent grain with a simple smooth shape in the same specimen formed no sharp GB. The results of this study indicate that the non-homogeneous stress/strain field is critical for the formation of the GB.