Shade from a tree or an artificial structure is not the only thing that can restrict plants' growth.
For instance, a garden in front of a north-facing building is not only in shade for most of the day, but also exposed to the prevailing winds. Plants that do not have adequate snow coverage in winter are particularly vulnerable to frost heave.
There are other problems involved in gardening in the shade of large trees. The roots of fast-growing trees such as willows, poplars, silver maples and Norway maples absorb much of the water and nutrients that other plants require.
Moreover, the ground under conifers is dry and acidic and gets little snow in winter to protect it.