QuestionNeed help on a mongo db project where code implementation is done… Need help on a mongo db project where code implementation is done using python sdkWeather Data Storage and Retrieval Introduction The weather data system supports storage of temperature and humidity values from various devices. It also has a device registry and user information. The data is stored in MongoDB. The focus of the project is to learn some data modeling in MongoDB and to implement various functionalities based on that. Each device sends either temperature or humidity data based on its type. There are two categories of users – admin and default. To some degree, this project is connected to the MongoDB database discussed in the Jupyter notebook for the weekly content on NoSQL. You will find some hints and relevant data modeling suggestions in that notebook. Housekeeping points ? This is a minimal example and may not follow some standard practices. ? We focus on the main flow, and not much error handling. ? To avoid handling command-line arguments, config file paths are hard-coded in the source files – not a general practice. Program Organization The simple program is structured in various layers. 1. Data model: a. users collection: This stores information about the users. It has username (String), email (String), and role (String) fields. b. devices collection: This stores information about the devices. It has device_id (String), desc (String), type (String – temperature/humidity) and manufacturer (String) fields. Each device generates only one type of data. c. weather_data collection: This stores the actual weather data. It contains device_id (String), value (Integer), and timestamp (Date) fields in each document. 2. src/setup.py: This program populates the initial data. a. First, drops the database to get rid of any existing data so that it can start afresh. b. Then creates a new database called weather_db c. Reads the user information from config/users.csv and populate those in the users collection d. Reads the device information from config/devices.csv and populate those in the devices collection e. Five temperature and five humidity sensors are created in the previous step. For weather data, this creates entries for each device in weather_data collection. It creates a data entry for each hour from 1st-5th December, 2020, for each device. The temperature and humidity values are randomly generated from a specific range. 3. src/database.py: This file hosts the Database class which provides base access to the MongoDB database. It has the connection information stored as static variables. It provides functionality for connecting, fetching, and inserting a document. 4. src/model.py: This file hosts UserModel, DeviceModel, and WeatherDataModel classes. a. UserModel: Provides functionality for finding a user by username or the auto-generated object id by Mongo. It also supports insertion which ensures that username remains unique. b. DeviceModel: Provides functionality for finding a device by device_id or the auto-generated object id by Mongo. It also supports insertion which ensures that device id remains unique. c. WeatherDataModel: Provides functionality for finding a data point by device_id and timestamp combination, or by the auto-generated object id by Mongo. It also supports insertion which ensures that data from the same device and the same timestamp is not duplicated. 5. src/main.py: Here, you need to add the functions required for the various problems. Initially, it contains examples of how to connect and use the various models to store and retrieve data. 6. docs/C01-Project-A-MongoDB-Prep-Material.ipynb: Contains documentation for various tasks that are helpful in the program. 7. docs/MongoDB-Install: Contains pointers to install both MongoDB and MongoDB Compass. Compass is a GUI that can help see and manipulate the various collections and their data. Problem Statement There are two enhancements to be done in this project. These involve both data modeling and relevant code additions. There are multiple ways to enhance the collections to solve these problems. You need to make decisions based on what might work best for the overall problem. You are free to choose the data structures and the output that you print, as long as they are sufficient to solve the problems and display the results. You also need to add to the underlying Model and Database classes to serve your new use cases. 1. (Medium) Add user access support for various data. a. You will notice that the role (admin/default) in the users collection is not actually being used to control access. You need to add support in all models to pass the username, and validate that only an admin role is allowed to store or retrieve any data. Assume that user credentials (password) validation has already happened outside of the system. b. Of course, the system cannot provide zero access to default role users. Now, you need to enhance the users collection to add support for devices that a user might have access to. Please see the notebook from the NoSQL week for hints on that. You need to provide control for either read (r) support, or both read/write (rw) support for multiple devices for each user in the users data model. You need to then add validations in both DeviceModel and WeatherDataModel to check if the user has access to perform the read or write operation, based on the devices they can access. For example: After updating the data model, say you add read access to DT001, and read/write access to DT002 for user user_1. Now user_1 should be able to read device and weather_data information for both DT001 and DT002. user_1 should also be able to write to device details and weather_data for DT002. The users collection should continue to be accessible only to admins as per 1.a In main.py, demonstrate support for 1.a and 1.b using specific calls to show both access successes and failures. You are free to specify any access mapping between users and devices to cover various cases. 2. (Hard) Add reporting support for weather data. There can be multiple data values inserted in each day for every device. We need to aggregate those to give an overall view for each day, per device. a. First, you need to design a new data model for a daily_reports collection. This should have support for storing daily aggregated data (average, minimum, maximum) for each device. Add a new DailyReportModel in model.py which should provide the base operations. b. Implement a data aggregator function which takes existing data from weather_data collection and adds the aggregated data in daily_reports collection. It should go over all the records of weather_data and aggregate per device per day. Normally, this would happen periodically on new data, but we will mimic it with a bulk aggregation function in this project. c. Implement a data retrieval function that takes a device id and a range of days, and returns average, minimum and maximum data values for each day, fetching data from the daily_reports collection.Computer ScienceEngineering & TechnologyPython Programming KLA MISC
