Rural Kalahari agro-pastoral communities of Southern Africa have been exposed to drought shocks throughout history and have adapted their livelihoods accordingly. Yet, drought continues to disrupt or threaten to disrupt their production systems. With semi arid Botswana as a case study, this paper hence sought to unearth the factors limiting agro-pastoral communities from adequately coping and adapting to drought. Low rainfall, which is also highly variable, coupled with relatively low soil fertility status make subsistence livestock keeping and crop cultivation risky. This marginal agricultural potential of the land is further compounded by other constraints. Some of the major constraints included persistence of droughts, limited diversification options outside agriculture, inadequate and poor quality drinking water (high salinity) for livestock, crop damage by wild animals as well as the current land tenure system which curtails the traditional response of livestock mobility during drought. All these factors may act solely or in combination to render rural communities vulnerable during droughts. It is therefore recommended that effective interventions be tailored to local conditions to enhance resilience among Kalahari’s rural population.