Simple Tips for Snowflake Data Engineers
Enable auto-suspend and auto-resume – Ensure that you set to suspend virtual warehouses automatically when processing is done. Auto-suspend will save credit consumption. Auto-resume will enable the warehouse after specific time limits.
Track costs by teams – Create roles by business functions to track spending by the line of business or by functional teams. You could use resource monitoring to cut off the data warehouses when a user hit predefined credit amount limits.
Pick the right region– Store your S3 buckets in the same geographic region where you are processing.
INSERT vs. COPY INTO – Instead of INSERT, use COPY INTO since it uses the more efficient for bulk loading operations.
Avoid ORDER BY – Sorting large results can be computationally expensive hence avoid using ORDER BY clauses in queries unless absolutely required.
Remove inactive users – Forrester says 80% of breaches are caused by excessive access privileges. You can avoid security and privacy incidents by removing inactive users. Protecto identifies inactive users, which helps you prevent big fines and security incidents.
Purge unused tables – Did you know 96% of data has not been used in the last 90 days? Dropping unused tables is a great way to keep your Snowflake instance clean. Protecto.ai finds unused data.