In the first part of the colloquium I will provide a gentle introduction to neural networks from a statistical-mechanics perspective. In this framework, a bridge between biologically-inspired models and artificial models is highlighted and leveraged to improve our comprehension and mathematical control on these systems. In the second part, I will show that consolidation and remotion mechanisms occurring in mammal’s brain during sleep can be recast into suitable machine-learning parameters and the hierarchical organisation of memories in the brain inspires a hierarchical architecture of layers in deep neural networks.