Design Rationale

This section provides insight into the key decisions made during the development of the prototype. For each decision, we explore the reasoning, alternatives considered, and their alignment with the project’s goals.

Decisions:

Decision 1: Choosing Python for Backend Development

Decision Made:

The backend of the renting website prototype is developed using Python.

Reasoning:

Python is known for its extensive support and libraries in machine learning and data processing, making it an ideal choice for developing simple or complex recommendation systems. Its simplicity and readability also contribute to easier maintenance and collaborative development. Moreover, Python’s widespread adoption in the tech community ensures a robust support network and continuous updates, which are vital for a project focusing on cutting-edge features.

Alternatives Considered:

Node.js: Known for its high performance and scalability in handling numerous connections simultaneously, making it a strong contender for backend development. However, it may not have as straightforward integration with machine learning libraries compared to Python.

Java: Offers robust performance and is widely used in large-scale systems. However, it does not provide the same ease of use for machine learning and data processing tasks as Python.

Ruby: Known for its easy syntax and rapid development capabilities, but it also lacks the extensive machine learning libraries and community support that Python offers.

Project Objectives Alignment:

Python’s strengths in machine learning and data processing align well with the websites need for an advanced recommendation system. Its ease of use and readability support collaborative development, which is crucial for a growing project. Python’s extensive libraries, such as scikit-learn and TensorFlow, provide the necessary tools to build and refine the recommendation algorithms, directly contributing to the project’s core mission.

Future Implications:

The choice of Python offers scalability and flexibility for future enhancements, especially in the realm of machine learning and data analytics. It ensures that the backend can be easily adapted and expanded as new features and algorithms are introduced. Additionally, Python’s popularity and large community support mean that emerging trends and advancements in technology can be readily integrated into the website.

Decision 2: Generating and Storing Sets of 30 Recommendations for Dynamic Updates

Decision Made:

The system generates and stores 30 recommendations at a time for each user. After the user has interacted with all 30 recommendations, their preferences are updated, and a new set of 30 recommendations is generated.

Reasoning:

This approach balances the efficiency of batch processing with the need for dynamic, real-time updates. By processing a set number of recommendations in batches, the system can manage computational resources more effectively. This number (30) is chosen based on the assumption that a user’s preferences are unlikely to change significantly within a smaller set of interactions, thus providing a more stable basis for generating relevant recommendations.

Alternatives Considered:

Real-Time Recommendation Generation: Generating recommendations in real-time for each user interaction. While this ensures the most up-to-date recommendations, it can be computationally intensive and may not significantly improve the user experience due to the relatively stable nature of user preferences.

Larger Batch Processing: Generating a larger batch of recommendations at a time. This could reduce the frequency of updates but might lead to less relevant recommendations over time as user preferences evolve.

Project Objectives Alignment:

This strategy aligns well with the core objective of providing relevant and personalized roommate recommendations. It ensures that the recommendations are updated frequently enough to reflect changing user preferences while maintaining system efficiency.

Future Implications:

This approach provides a scalable framework for the recommendation system. It allows for adjustments in the batch size based on future insights into user behavior and system performance. The balance between batch processing and real-time updates offers flexibility to adapt to changing user needs and technological advancements. However it is worth noting that each time we have to fetch another set of 30 recommendations we must re-filter the set of users to get candidates and then recalculate dot products for cosine similarity which may not be the most efficient. In the future when the user base grows too large we may have to strike a balance between up to date exact recommendations vs approximate recommendations where we generate recommendations in a large batch and make small updates as we go.

Decision 3: Using Cosine Similarity for Recommendation Metrics

Decision Made:

Implementing cosine similarity as the primary metric for collaborative and content-based filtering in the recommendation system.

Reasoning:

Cosine similarity is particularly effective for high-dimensional data, common in recommendation systems. It measures the cosine of the angle between two vectors, in this case, user preference or feature vectors, which is a robust indicator of similarity in user preferences or characteristics. This metric is advantageous for its insensitivity to the magnitude of the vectors, focusing solely on orientation, which is ideal for comparing user profiles.

Alternatives Considered:

Euclidean Distance: While useful for measuring direct distances between points, it can be less effective in high-dimensional spaces where the notion of ‘distance’ becomes less intuitive.

Pearson Correlation: Measures linear correlation between two variables but may not perform as well in scenarios where the relationship is not linear.

Jaccard Index: Useful for measuring similarities between sets, but less effective for the type of numerical, high-dimensional data typically found in user profiles.

Project Objectives Alignment:

Using cosine similarity aligns with the objective of creating an effective and user-friendly roommate matching system. It ensures that the recommendations are based on meaningful similarities in user preferences and characteristics, leading to more compatible matches.

Future Implications:

The choice of cosine similarity lays a solid foundation for the recommendation algorithm, capable of handling diverse and complex user data. It offers a scalable metric that can adapt to an expanding feature set and user base, ensuring the long-term efficacy of the recommendation system.

Decision 4: Calculating Cosine Similarity in the Database for Efficiency

Decision Made:

Transitioning from calculating cosine similarity in Python to directly computing the dot product in the database after L2 normalization of user data.

Reasoning:

Concurrency Issues with Global Variables: Initially, using sparse matrices in Python and storing them as global variables offered efficiency but led to concurrency issues. Multiple simultaneous recommendation requests accessing the same global variables could lead to conflicts or race conditions.

Overhead in Data Transfer to Python: Shifting to fetching necessary data from the database for each recommendation request and calculating cosine similarity in Python introduced significant data transfer overhead, impacting system performance.

Efficiency of Database Computation: Ultimately, preprocessing data for L2 normalization and calculating the dot product directly in the database was chosen. This approach significantly reduces data transfer overhead and leverages the database’s optimized computation capabilities, leading to faster and more efficient recommendation generation.

Alternatives Considered:

Maintaining Sparse Matrices in Python: Despite its efficiency for single requests, this approach’s limitations in handling concurrent requests made it less viable for a scalable system.

On-the-fly Calculation in Python: While offering flexibility in data processing, the added latency due to data transfer proved to be a significant bottleneck.

Project Objectives Alignment:

This decision aligns with the project’s goal of providing efficient and responsive roommate matching services. By optimizing the recommendation algorithm for concurrency and minimizing data transfer overhead, the system can handle multiple requests simultaneously without performance degradation.

Future Implications:

This approach sets a scalable foundation for the recommendation system. It accommodates an increasing user base and the accompanying rise in concurrent requests. Moreover, database-level computations ensure that the system can adapt to evolving data structures and user interaction patterns while maintaining performance.

Decision 5: Database Connection Pooling

Decision Made:

Implementing a database connection pool using PostgreSQL and Python’s psycopg2 library.

Reasoning:

Database connection pooling is crucial for managing multiple simultaneous database connections efficiently. By maintaining a pool of connections, the system can reuse existing connections, reducing the overhead of establishing a new connection for each database query. This approach significantly improves the application’s performance, especially under heavy load. PostgreSQL, known for its reliability and robustness, was chosen as the database system, and Python’s psycopg2 library was selected for its compatibility and ease of integration with PostgreSQL.

Alternatives Considered:

Single Connection per Request: Each request establishes and closes its database connection. While simple, this method is inefficient under high load, leading to increased latency and resource consumption.

Other Database Systems: Alternative databases like MySQL or NoSQL databases like MongoDB were considered. However, PostgreSQL was preferred for its advanced features, reliability, and support for complex queries and data types.

Project Objectives Alignment:

Implementing database connection pooling aligns with the need for a scalable, efficient backend system capable of handling multiple user requests simultaneously. It ensures that the system remains responsive and efficient as the user base grows.

Future Implications:

This decision provides a scalable infrastructure foundation, allowing for efficient management of database connections as the application grows. It also opens up possibilities for future enhancements in database management and optimization.

Decision 6: User Authentication Method

Decision Made:

Choosing Firebase for user authentication and session management.

Reasoning:

Firebase was selected for its comprehensive set of features, including authentication, real-time database, and cloud functions. It provides a secure and scalable solution for managing user sessions and authentication across different platforms. Firebase’s ease of integration with various front-end and back-end technologies makes it a versatile choice for the project.

Alternatives Considered:

Custom Authentication System: Building an in-house authentication system using tokens or session-based authentication. While offering control and customization, it requires significant effort to ensure security and scalability.

OAuth Providers: Using third-party OAuth providers like Google or Facebook for authentication. While easy to implement, relying solely on third-party providers can limit control and flexibility in user management.

Project Objectives Alignment:

The use of Firebase aligns with the project’s objectives of providing a secure, user-friendly, and seamless experience. Its integration with the backend and frontend ensures a consistent authentication flow, crucial for user trust and engagement.

Future Implications:

Firebase’s scalability and robustness offer a future-proof solution for user authentication and session management. Its flexibility allows for easy integration of additional authentication methods and features, catering to the evolving needs of the application and its users.

Decision 7: Flask as the Web Framework

Decision Made:

Selecting Flask to develop the web application portion of the renting website prototype.

Reasoning:

Flask, a lightweight and flexible Python web framework, was chosen for its simplicity and adaptability. Its minimalist nature allows for greater control over the application structure and the choice of components, making it suitable for a project that requires a tailored approach. Flask’s extensive compatibility with various libraries and tools in the Python ecosystem, including machine learning and database interaction libraries, played a significant role in this decision.

Alternatives Considered:

Django: A more feature-rich and opinionated web framework. While Django offers a comprehensive set of tools and a built-in ORM, its complexity and rigid structure could have constrained the custom development needed for the renting website.

Node.js with Express: Offers high performance and is well-suited for real-time applications. However, it would require a shift from Python, which is central to the project’s machine learning and data processing capabilities.

Project Objectives Alignment:

Using Flask aligns with the project’s need for a customizable, efficient, and scalable web framework that seamlessly integrates with the Python-based backend logic. Its lightweight nature ensures that the focus remains on the application’s unique functionalities without unnecessary overhead.

Future Implications:

Choosing Flask provides a solid foundation for the web application, ensuring scalability and flexibility. As the project evolves, Flask’s adaptability will allow for the integration of new features and technologies without significant constraints.

Decision 8: Data Structure for User Preferences and Features

Decision Made:

Developing a structured format for storing user preferences and features in the PostgreSQL database using JSONB and relational data models.

Reasoning:

The choice to use JSONB and relational data models was driven by the need for flexibility in storing and querying diverse and dynamic user data. JSONB format in PostgreSQL allows for efficient storage of unstructured data like user preferences and features, which can vary significantly between users. This format supports quick retrieval and easy updates, which are crucial for the recommendation system. Additionally, the use of a relational data model for structured data ensures data integrity and efficient querying for standard user attributes.

Alternatives Considered:

Fully Normalized Relational Model: While offering benefits in data integrity and normalization, it could become complex and inefficient for handling dynamic and diverse user data like preferences and features.

NoSQL Databases: These databases excel in handling unstructured data but might lack in transactional integrity and complex querying capabilities compared to relational databases like PostgreSQL.

Project Objectives Alignment:

This data structure approach supports the project’s goal of providing personalized and adaptable roommate matching services. The flexibility in handling user data variations is crucial for the recommendation system’s effectiveness and responsiveness to user preferences.

Future Implications:

The chosen data structure offers scalability and adaptability for future developments. As user data and application requirements evolve, this approach will accommodate changes in data types and structures, ensuring the system remains robust and efficient.

Decision 9: Handling User Interactions

Decision Made:

Developing a system within the renting website prototype to record and process user interactions, such as likes, dislikes, and superlikes.

Reasoning:

The decision to create a robust system for tracking user interactions was driven by the need to understand user preferences and behaviors accurately. This system includes a dedicated database schema to record every interaction. Each interaction is stored with details including the user IDs involved and the type of interaction. This data is crucial for refining user recommendations and enhancing the matching algorithm.

Alternatives Considered:

Simpler Logging Mechanism: Only tracking the number of interactions without detailed records. While less resource-intensive, this approach would provide less insight into user behavior and preferences.

Third-Party Analytics Services: Utilizing external services for interaction tracking. This option could simplify development but might limit customization and data control.

Project Objectives Alignment:

Recording detailed user interactions aligns with the project’s core objective of delivering personalized and relevant roommate recommendations. This system enables the application to adapt recommendations based on real user feedback, enhancing the matching accuracy.

Future Implications:

This approach lays the groundwork for a data-rich environment conducive to machine learning and data analysis. Over time, the accumulated interaction data can be leveraged for more sophisticated analytics and recommendation strategies, aligning with the project’s long-term vision for adaptability and user-centric enhancements.

Decision 10: Recommendation Update Strategy

Decision Made:

Establishing a strategy for updating user recommendations in the renting website prototype based on user interactions, balancing between real-time updates and computational efficiency.

Reasoning:

The chosen strategy involves updating recommendations in a semi-batch process. After a user interacts with a predetermined number of recommendations (e.g., 30), the system updates their preference profile and generates a new set of recommendations. This approach offers a middle ground between the responsiveness of real-time updates and the efficiency of batch processing, ensuring recommendations remain relevant without overloading the system.

Alternatives Considered:

Real-Time Updates: Updating recommendations immediately after each interaction. While highly responsive, this could lead to significant computational overhead and might not significantly enhance user experience due to the relatively stable nature of preferences.

Larger Batch Processing: Updating recommendations after a larger set of interactions. More efficient but potentially less responsive to quick changes in user preferences.

Project Objectives Alignment:

This strategy aligns with the objective of providing personalized and adaptive roommate matching services. It ensures that recommendations are refreshed frequently enough to reflect evolving user preferences while maintaining operational efficiency.

Future Implications:

The chosen update strategy provides a scalable framework for the recommendation system. As the user base grows and technological capabilities evolve, the strategy can be adjusted for optimal balance between responsiveness and efficiency. This approach also opens avenues for implementing more advanced machine learning techniques that can further refine the recommendation process based on user interactions.

Decision 11: Image Processing for User Profiles

Decision Made:

Implementing a strategy for handling user profile images within the renting website prototype, which includes resizing, cropping, and appropriate storage.

Reasoning:

The decision to incorporate a robust image processing system for user profiles was driven by the need for consistency in user experience and efficient use of server resources. This includes automating the process of resizing and cropping images to a standard format and size, ensuring uniformity across all user profiles. Efficient image processing also contributes to faster page load times and a smoother user interface.

Alternatives Considered:

Client-Side Image Processing: Allowing users to edit and resize images on their end before uploading. While this approach could reduce server load, it might lead to inconsistencies in image quality and format.

External Image Processing Services: Using third-party services for image processing. This option could offload computational work from the server but might increase dependencies and raise concerns about data privacy.

Project Objectives Alignment:

Having a consistent and efficient image processing system aligns with the project’s goal of providing a seamless and visually appealing user interface. It enhances user experience by ensuring that profile images across the platform are standardized and of high quality.

Future Implications:

This decision ensures the scalability of the image handling process as the user base grows. It allows for potential future enhancements, such as implementing more advanced image processing techniques or integrating AI-based features like facial recognition to improve user profile authenticity.

Decision 12: Front-End to Back-End Communication

Decision Made:

Establishing a structured approach for front-end to back-end communication in the renting website prototype, focusing on efficient API design and standardized data exchange formats.

Reasoning:

A well-defined communication protocol between the front-end and back-end is crucial for the smooth operation of the application. This involves designing RESTful APIs that handle requests and responses in a standardized format, typically JSON. This approach ensures that data exchange is efficient, secure, and easily manageable. The use of RESTful principles also facilitates scalability and maintainability of the application.

Alternatives Considered:

GraphQL: An alternative to RESTful APIs, offering more flexibility in querying data. However, it might introduce complexity in implementation and require additional learning for the development team.

WebSocket Communication: Ideal for real-time data exchange but might be overkill for the current scope of the application and could introduce complexity in handling stateful connections.

Project Objectives Alignment:

Adopting RESTful APIs for communication aligns with the project’s objectives of creating a scalable, efficient, and user-friendly platform. Standardized data exchange formats ensure consistency across different parts of the application and facilitate easier debugging and testing.

Future Implications:

This decision provides a solid foundation for future development and expansion of the renting website prototype. It allows for easier integration of new features and modules, as well as potential scaling to accommodate a growing user base. The choice of RESTful APIs also ensures compatibility with a wide range of tools and technologies, offering flexibility for future technological advancements.

Conclusion

The design decisions made during the development of the renting website prototype reflect a thoughtful and strategic approach to creating a robust and scalable roommate matching platform. Each decision, from choosing Python for backend development to establishing efficient communication between front-end and back-end, was guided by a commitment to achieving the project’s core objectives: providing a user-friendly, efficient, and adaptable service.

See Also