<h1 id="data-lake">Data Lake<a aria-hidden="true" class="anchor-heading icon-link" href="#data-lake"></a></h1>
<p>Data lakes are large repositories of data in their natural storage or transformed formats. They're often used as the data source for analytics, reporting, data visualisations and machine learning models.</p>
<p>Data can be in just about any format:</p>
<ul>
<li>Structured</li>
<li>Semi-structured</li>
<li>Unstructured</li>
</ul>
<h1 id="generations">Generations<a aria-hidden="true" class="anchor-heading icon-link" href="#generations"></a></h1>
<p>Gen1 was built independently of Storage. Gen2 was built on on blob storage and introduces "Hierarchical namespaces", which replace the folder emulation in blob containers with a true directory structure. This allows for transactions at the folder level and enforcement of <abbr title="Portable Operating System Interface">POSIX</abbr> permissions.</p>
<p>The long term intention appears to be that all features should be supported with this feature enabled, though that isn't the case at the present time. The data is accessible via multiple APIs: both <code>blob.core.windows.net</code> and <code>dfs.core.windows.net</code> endpoints are exposed for storage accounts with <abbr title="Hierarchical Namespace">HNS</abbr> enabled.</p>

Data Lake


![Digital brain](assets/images/logo.svg){display: block, margin: 0 auto, max-height: 50vh}

Hi, I'm Luke 👋

I'm a software engineer turned site reliability engineer. I'm currently big on knowledge management and learning.

# Find me elsewhere

- [👨‍💻 DEV](https://dev.to/lukecarrier)
- [🚢 GitHub](https://github.com/LukeCarrier)
- [👔 LinkedIn](https://www.linkedin.com/in/lukecarrier/)
- [📄 Résumé](https://github.com/LukeCarrier/resume)
