Add niconico video Provider (#2339)
* Initial comit of NicoNico provider
* Remove unwanted logout calls
* niconico: Split and organize classes
* Add artist retrieval method and fix Explorer mixin imports in Niconico provider
* Niconico: Enhance artist parsing with metadata and SNS links
* Niconico: Add sensitive content handling and improve recommendations feature
* Niconico: Implement series handling as albums and enhance library retrieval methods
* Niconico: Refactor the structure of the Nikoniko adapter and split the individual adapters into separate files.
* Niconico: Add URLs for playlists, tracks, and artists in parsers; update import in album mixin
* Refactor Niconico provider mixins and add configuration management
- Enhanced NiconicoMusicProviderExplorerMixin to fetch recommendations with filtering based on user history, following activities, and like history.
- Introduced NiconicoConfig class for managing provider-specific configurations, including recommendation counts and tag filtering.
- Implemented TagManager for caching and retrieving video tags asynchronously, with support for deduplication and periodic cleanup.
- Updated library mixin to support adding and removing tracks with auto-like functionality.
- Improved playlist mixin to handle adding and removing tracks from playlists with error handling.
- Added support for fetching and filtering tracks based on required tags in the track mixin.
- Created a new configuration file for Niconico provider settings.
* Niconico: Update workspace configuration to include additional paths and launch settings
* Niconico: Improve descriptions in configuration entries for clarity and consistency
* Niconico: Simplify get_artist_toptracks method to retrieve the newest 50 tracks
* Niconico: Add auto-sync and include library track artists settings; refactor library mixin
* Niconico: Refactor search methods for improved clarity and efficiency; update configuration entries for email and sensitive content handling
* Refactor Niconico search and user adapters; update parsing methods for improved track retrieval and error handling
* Niconico: Add methods for retrieving own videos and series; update configuration entries for improved content management
* Niconico: Enhance search and recommendation features; refactor configuration and constants for improved tag handling
* Refactor video parsing methods to handle None values and skip muted videos for improved track retrieval
* Remove Niconico server workspace configuration file
* Refactor Niconico provider: Enhance error handling, logging
- Replaced call_with_throttler with _call_with_throttler in NiconicoVideoAdapter for consistency.
- Removed handle_niconico_errors context manager and replaced it with direct logging in various mixins.
- Introduced log_verbose and log_verbose_operation functions for improved logging.
- Updated methods across mixins to utilize new logging functions and streamline error handling.
- Refactored tag management to improve caching and fetching logic.
* Niconico: Update icon SVG files for improved styling and add monochrome version
* Niconico: Renamed parsing to conversion functions for playlists, tracks, and series
* Refactor Niconico video adapter and track mixin: Improve type hints, enhance error handling, and streamline stream format processing
* Refactor type hints in adapter
* Renamed the provider from “niconico” to “nicovideo”.
* Nicovideo: Enhance cache management by limiting concurrent operations to reduce database load
* Nicovideo: Include own uploaded videos in library track retrieval
* Remove unused sensitive contents configuration from Nicovideo provider
* Update NicoVideo monochrome icon SVG with new design and filter effects
* Nicovideo: Add icons for various sections in the music provider explorer mixin
* Nicovideo: Improve tag fetching with async task locking to prevent race conditions
* Nicovideo: Refactor provider mixins and enhance core functionality with MixinCaller
* Nicovideo: Adjust low priority throttler period for improved background tag updates
* Refactor Nicovideo provider: enhance video fetching and tag management, remove TagManager
* Refactor logging in Nicovideo provider: Adjusted log levels.
* Nicovideo: Enhance error logging for ValidationError in API calls
* Refactor Nicovideo provider mixins: streamline library item retrieval and enhance caching logic
* Refactor Nicovideo Adapter Structure and Update Mixins
- Introduced a new NicovideoAdapterHub to centralize adapter functionality.
- Updated mixins to use the new adapter hub instead of individual adapters.
- Implemented utility functions for common operations across converters.
- Improved metadata handling and mapping for media items.
- Ensured compatibility with the latest niconico API changes.
* Refactor NicovideoTrackConverter: Move audio format creation logic to a dedicated method and remove unused utility function
* Refactor Nicovideo provider mixins: rename mixin files.
* Renamed from adapter to service.
* Refactor Nicovideo service architecture: rename Hub to Manager
* Add override decorator to get_stream_details method in NicovideoMusicProvider
* Add override decorators to library methods in artist and track mixins
* Refactor Nicovideo converters: replace utility functions with helper methods
* Remove unused logging import from album mixin
* First implementation of Nicovideo testing.
* Refactor track handling in NicovideoMusicProviderTrackMixin to improve stream metadata retrieval
* requirements: add yt-dlp version 2025.6.30 to dependencies
* fix: update artist mapping to return UniqueList for better handling
* feat: add DUMMY_IS_PEAK_TIME constant and update helper functions to use it; update requirements to include yt-dlp
* feat: update yt-dlp version to 2025.8.11 in manifest and requirements
* feat: update workflow and main.py to latest versions from dev and fixture-updates branches
* Refactor fixture data generation and saving process
- Updated JSON fixture files with dummy descriptions for testing purposes.
- Enhanced helper functions for consistent snapshot comparison and dynamic field stabilization.
- Introduced new scripts for generating and saving fixture data with integrated diff tracking.
- Improved logging for fixture generation and changes.
* fix: update schedule comment to reflect daily execution and enhance condition for fixture updates
* fix: update descriptions in test snapshots to use dummy text for consistency
* fix: remove obsolete check-test-session script
* fix: update release dates and dummy data in test fixtures for consistency
* fix: enhance stabilization logic by consolidating dynamic field and count value processing
* fix: streamline provider unload logic and improve error handling
* Refactoring: refactor niconico service methods to unify series and mylist retrieval logic.Made niconico_py_client accessible via properties in the service.
* Update fixtures-update-and-test.yml
* Refactor: streamline audio format handling by introducing create_audio_format function and centralizing constants
* Refactor: improve stream details handling by safely retrieving HTTP headers and simplifying audio format parameters
* update snapshot
* Fix: Add “extra_input_args” from streamdetails to get_preview_stream and adjust additional input arguments.
* Update fixtures-update-and-test.yml
* Refactoring: Added a stream converter to enhance the nicovideo provider and improve stream detail processing. Removed dependency on yt-dlp.
* - add StabilizationInfo
- Add Stream Test
- Called ruff after automatic generation.
* Fix: Update comment to clarify usage of StreamConversionData serialization in import statement
* Fix: Update media type checks in artist and track mixins for consistency
* Fix: Update search_videos_by_tags method to accept a single tag and streamline video search logic
* Refactoring: Remove unused BaseModel serialization strategy and related imports from helpers.py
* Fix: Rename search_videos_by_tags method to search_videos_by_tag for consistency
* Fix: Update error messages in artist mixin for clarity and detail; add note on playlist duplicate entries
* Fix: Enhance logging with safe argument summaries and caller information extraction
* Fix: Update niconico.py requirement to specific commit for stability
* Fix: Remove follow/unfollow artists functionality from Nicovideo configuration and artist mixin
* Fix: Removed logout upon unloading, and prevented attempts to log in again when authentication information is not available.
* Fix: Remove unnecessary login checks in various mixins and services to streamline data retrieval
* Fix: Enhance album mapping creation with optional thumbnail URL and update artist yield logic to handle ItemMapping conversion
* Fix: Changed to directly generate album and artist mappings.
* Fix: Changed to directly generate album and artist mappings.
* Fix: Update label for user session configuration to clarify cookie usage
* Fix: Remove stage from niconico video manifest
* Fix: Update workflow description to clarify NICONICO_SESSION requirement
* Fix: Rename nicovideo provider fixtures update and test workflow
* Fix: Simplify docstring in get_config_entries and add category comments in config entries
* Fix: Refactor get_config_entries_impl to separate config entry categories and simplify parameters
* Move :nicovideo workflows
* Fix: Update conversion function for SeriesData to handle album tracks
* Fix: Update usage instructions for fixture generation script to reflect correct environment variable
* Fix: Remove unused convert_to_netscape function and related imports
* Fix: Refactor configuration constants for nicovideo provider by moving keys to config.py and cleaning up imports
* Fix: Remove unused DUMMY_DESCRIPTION constant and update stabilization rule for description
* Refactor Nicovideo provider configuration system to use category-based structure
- Introduced a new configuration system for the Nicovideo provider, organizing settings into categories: Auth, Content, and Recommendations.
- Updated all relevant code to access configuration values through the new structure, replacing old method calls with direct attribute access.
- Enhanced clarity and maintainability of the configuration management by encapsulating related settings within dedicated classes.
* Fix: Move recommendation_filter_tags configuration to the correct position in RecommendationsConfigCategory
* Fix: Update recommendation configuration keys and labels for clarity and consistency
* Fix: Add integrated loudness to stream conversion output
* update snapshot loudness
* Add: Added dependency on niconico.py for lint testing.
* Fix: Update niconico.py dependency PEP 508 format in manifest and requirements files
* Remove unused get_supported_features_for_mixin methods
- Remove get_supported_features_for_mixin() from all mixins (base, artist, track, album, playlist, explorer)
- Remove unused ProviderFeature imports
- Features are now statically defined in __init__.py's SUPPORTED_FEATURES
* Simplify config: Remove Content and Recommendations categories
This change addresses PR review feedback requesting a simpler configuration.
The complex Content and Recommendations config categories have been removed
and replaced with sensible defaults:
- Content defaults: Include own mylists, exclude followed mylists
- Recommendations defaults: 25/50/30 track counts, no tag filtering
- Auto-like: Enabled by default
- Sensitive content: Mask by default
The removed complex configuration has been preserved in the
`add-nicovideo-advanced-options` branch for potential future enhancement.
This makes the initial provider setup much simpler while maintaining
core functionality.
* Refactor: Add @override, remove unused filtering, use SENSITIVE_CONTENTS constant
- Add missing @override decorator to get_library_playlists
- Remove _fetch_similar_tracks_with_filtering (no longer filters anything)
- Extract SENSITIVE_CONTENTS constant to nicovideo/constants.py
- Replace all hardcoded sensitive_contents with constant reference
These changes improve code clarity and maintainability.
* Update niconico.py dependency to the latest commit
* refactor: Simplify mixin architecture - remove MixinCaller abstraction
- Remove MixinCaller class with complex lambda-based invoke methods
- Replace with explicit for loops in provider methods
- Make class inheritance explicit (list 6 mixin classes directly)
- Convert provider_mixins/__init__.py to static __all__ list
- Move NICOVIDEO_MIXINS constant to provider.py
- Change reversed() to [::-1] for type checker compatibility
This addresses PR feedback about code complexity while maintaining
mixin-based separation of concerns.
* refactor: Remove provider-specific pre-commit hook
Remove check-nicovideo-test-session hook to maintain provider isolation.
Instead, enforce environment variable usage in fixture generation script:
- Require NICONICO_SESSION environment variable (no hardcoded fallback)
- Add clear error message when variable is not set
- Update documentation to reflect environment-only authentication
This prevents accidental credential commits while keeping the provider
isolated from repository-wide configuration.
* test: Update converter snapshots for new model fields
Update test snapshots to include new fields added to Music Assistant models:
- grouping: Album grouping field
- in_library: Library membership flag
* fix: nicovideo fixtures workflow - use pip instead of uv
- Remove dependency on uv and setup.sh
- Use standard pip installation like test.yml
- Remove unnecessary venv caching
- Simplify workflow by using GitHub Actions default Python environment
* fix: update niconico.py commit hash to latest (
711729d)
- Align pyproject.toml with requirements_all.txt
- Fix dependency conflict in GitHub Actions
* refactor: remove nicovideo from mypy exclude list
- nicovideo provider is fully mypy strict mode compliant
- Dependencies managed via manifest.json and requirements_all.txt
- Align with other strict-mode compliant providers (spotify, deezer, etc.)
* fix: resolve mypy strict mode errors in nicovideo provider
- Add proper TYPE_CHECKING import for pydantic BaseModel
- Add explicit type casts to resolve no-any-return errors
- Import Any type for type annotations in tests
- Ensure all return statements have explicit types
* fix: add niconico.py to test dependencies for proper mypy type checking
- Add niconico.py to project.optional-dependencies[test]
- Revert manual type cast workarounds
- Ensures pydantic types from niconico.py are properly resolved during CI
- Simpler and more maintainable than type annotation workarounds
* fix: remove deprecated enable_cache parameter from StreamDetails
- Remove enable_cache parameter removed in music-assistant-models v1.1.61
- This fixes mypy errors after merging upstream/dev changes
* nicovideo snapshots update
* refactor(nicovideo): implement fast HLS seeking with StreamType.CUSTOM
Replace StreamType.HTTP with CUSTOM for optimized seeking support.
Key improvements:
- Parse m3u8 once during stream conversion (not per seek)
- Generate dynamic m3u8 starting from seek segment
- Separate responsibilities: fetch → parse → generate → stream
- Use VERBOSE logging for seek operations
- Organize helpers into directory structure
* nicovideo update test,fixture,snapshot
* Remove FFmpeg submodule directory
* refactor: remove get_stream_details_for_mixin pattern, use normal override in TrackMixin
* docs: add detailed comments explaining StreamType.CUSTOM seek optimization
- Add comprehensive explanation in stream.py for why CUSTOM is used
- Document two-stage seek optimization (segment selection + input-side -ss)
- Explain input-side -ss limitation (can't cross segment boundaries)
- Add stage-by-stage comments in get_audio_stream and create_stream_context
- Clarify performance trade-off vs output-side -ss
* refactor(nicovideo): improve HLS streaming code structure and naming
Major changes:
- Rename NicovideoHLSProcessor → HLSSeekOptimizer for clarity
- Extract HLS data models to separate hls_models.py file
- Standardize m3u8 → playlist terminology throughout codebase
- Add HLS prefix to public API names to avoid confusion with track playlists
- Introduce DOMAND_BID_COOKIE_NAME constant for authentication
- Update method names for better semantic clarity:
* _get_stream_data → _prepare_conversion_data
* convert_by_stream_data → convert_from_conversion_data
* _fetch_hls_m3u8_text → _fetch_media_playlist_text
* _serve_m3u8 → _serve_hls_playlist
File structure:
- hls_processor.py → hls_seek_optimizer.py (renamed)
- hls_models.py (new): ParsedHLSPlaylist, HLSSegment
All tests and fixtures updated accordingly.
* refactor(nicovideo/tests): improve test module architecture
- Simplify TypeToConverterMappingRegistry (remove 3 unused methods)
- Extract all stabilization logic into FieldStabilizer class
- Add helper method to ConverterTestRunner for clarity
- Document FixtureProcessorProtocol design intent
Reduces complexity while maintaining type safety and auto-update features
* refactor(nicovideo/tests): improve naming and separate concerns
Renamed classes for clarity:
- TypeToConverterMapping → APIResponseConverterMapping
- FixtureManager → FixtureLoader
- FixtureDataGenerators → APIFixtureCollector
- FixtureDataSaver → FixtureSaver
- PathToTypeMapper → FixtureTypeMappingCollector + FixtureTypeMappingFileGenerator
Changes:
- Updated method names: generate_* → collect_* in APIFixtureCollector
- Fixed type completion by changing list to tuple for API_RESPONSE_CONVERTER_MAPPINGS
- Removed re-export pattern from fixtures/__init__.py
- Added comprehensive documentation to test_converters.py
Benefits: clearer responsibilities, better type inference, improved maintainability
* fix: update niconico.py dependency to latest commit
* Refactor: Reorganize fixture structure and update documentation
- Move fixture generation to separate repository (music_assistant_nicovideo_fixtures)
- Restructure fixtures: use fixture_data/ for generated data
- Update fixture loading to use new structure with fixture_type_mappings
- Remove embedded fixture generation scripts from tests
- Add comprehensive README for test directory
- Update VS Code tasks for easier fixture updates
- Delete obsolete GitHub workflow for fixture generation
* refactor: apply @use_cache to nicovideo provider methods
- Apply @use_cache decorator to get_* methods for improved performance
- Remove manual cache handling (cache_track helper and complex album info updates)
- Adjust cache durations based on SoundCloud patterns:
- get_artist: 14 days (was 30 days)
- get_track: 14 days (was 7 days)
- get_playlist: 14 days (was 30 days)
- get_playlist_tracks: 3 hours (unchanged)
- search: 3 hours (was 1 hour)
- get_album methods: 7 days (unchanged)
- Simplify code by leveraging Music Assistant's standard caching system
- Fix imports and add proper noqa comments for @use_cache usage
* revert: remove VS Code launch configuration changes
Remove the 'Music Assistant: Snapshot Update' launch configuration that was
added in previous commits. As requested in PR review, this change belongs
in a separate PR focused on development tools rather than the nicovideo
provider implementation.
* refactor: remove inappropriate cross-controller iteration
Address PR feedback by removing methods that iterate other controllers:
- Remove get_library_tracks and get_library_albums methods
- Remove library_add_for_mixin and library_remove_for_mixin methods
- Remove get_library_artists iteration over tracks controller
- Remove corresponding ProviderFeature flags from SUPPORTED_FEATURES
- Fix imports and TYPE_CHECKING blocks
The provider now only handles its native concepts (followed artists,
mylists, series) without inappropriately accessing other controllers.
This aligns with Music Assistant's architecture where providers should
only return in-library or favorited items from their own domain.
* fix: remove unintended cache category from get_track
Remove category=1 parameter from @use_cache decorator in get_track method
to maintain consistency with other cached methods that use default
category 0. This aligns with the pattern used across other providers
and removes unnecessary complexity.
* fix: Move media item types out of TYPE_CHECKING to fix @use_cache runtime errors
- Move Album, Artist, Track imports from TYPE_CHECKING to regular imports in artist.py
- Add noqa: TC002 comments for @use_cache decorator compatibility
- Format media_items imports in track.py consistently
- Fixes NameError: name 'Artist' is not defined when using @use_cache decorator
- Aligns with other Music Assistant provider patterns
* refactor: improve get_following_activities and fix throttler settings
- Call get_track directly instead of get_provider_item for simplicity
- Remove unnecessary semaphore (throttler already handles rate limiting)
- Fix throttler settings: rate_limit=5, period=1 (5 req/sec) for high priority
- Fix low priority throttler: period=1 for consistency
* refactor: optimize get_following_activities to use direct Activity conversion
- Add convert_by_activity() method to track converter for lightweight feed conversion
- Replace get_track() loop with direct Activity-to-Track conversion
- Remove unnecessary asyncio.gather overhead
- Use ItemMapping for artist info instead of creating Owner objects
- Improves performance by reducing API calls from N+1 to 1
* Update test snapshots for upstream StreamDetails changes
* feat: switch from Git to PyPI reference for niconico.py-ma
- Update pyproject.toml to use niconico.py-ma==2.1.0.post1 from PyPI
- Update requirements_all.txt with PyPI package reference
- Update nicovideo provider manifest.json
- Update fixtures project dependency
This addresses PR feedback to use PyPI packages instead of Git references.
* chore: update fixture data docstring from generator
- Minor docstring update from fixture generation process
- No functional changes
* refactor(nicovideo): address Copilot review feedback
- Simplify to_dict_for_snapshot to assert dict return type
- Switch auth service logging to lazy % formatting
- Fix double space in recommendation folder label
- Document 'additionals' in codespell ignore list
* chore: remove niconico.py-ma dependency from test requirements
* Reimplement get_own_followings to fetch followed artists
Restore the ability to retrieve users that the current user is following,
which was removed during config simplification.
* Remove unused user service methods
Remove methods that are no longer used in the current MA architecture:
- get_following_mylists() - empty stub returning []
- get_following_playlists() - only called get_following_mylists
- follow_user() - no MA interface to call this
- unfollow_user() - no MA interface to call this
These were remnants from config simplification that have no callers.
* Remove unused methods from nicovideo implementation
- Remove NicovideoUserService.get_own_videos()
- Remove NicovideoVideoService.like_video()
- Remove NicovideoSeriesService.get_own_series()
- Remove NicovideoSearchService.search_videos_by_tag()
- Remove unused imports (VideoSearchSortKey, VideoSearchSortOrder)
* refactor: simplify throttler implementation in nicovideo provider
- Use existing helpers/throttle_retry.ThrottlerManager instead of custom implementation
- Remove unused priority control (ApiPriority.LOW was never used)
- Remove verbose logging (sanitization, argument summaries, success logs)
- Keep only error logging with caller info for diagnostics
- Align with other providers' logging patterns (Spotify, Qobuz, Tidal)
* docs: Add explanation for 'Domand' spelling in niconico API
Clarify that 'Domand' is the actual spelling used in niconico's API
(not a typo). This naming appears in API endpoints and throughout
their media delivery system classes.
* docs(nicovideo): clarify HLS seeking limitation in stream.py
Improve technical accuracy of comment explaining why input-side -ss
fails with HLS without playlist reconstruction.
Previous comment incorrectly suggested the issue was about 'crossing
segment boundaries', but the actual problem is that FFmpeg cannot
identify target segments before parsing the playlist structure.
This clarification better reflects the implementation constraint.
---------
Co-authored-by: Shi-553 <58556078+Shi-553@users.noreply.github>
YTM: Yield control to event loop during stream url resolution to prevent blocking the main thread.
Update Resonate provider with `MULTI_DEVICE_DSP` support and other improvements (#2616)
* Update `MediaStream` usage
* Support `MULTI_DEVICE_DSP` for resonate
* Rename `stream` to `channel` in `MediaStream`
* Initial working implementation for per player DSP support
* Refactors, fixes, and comments
* Rename `shared_buffer` to `chunk_buffer`
* Use `UUID` instead of `str`
* Use seconds in `multi_client_stream`
* Move attribute definitions to class level
* Remove dead code
* Refactor `_cleanup_old_chunks`
* Rename `MultiClientStream` to `TimedClientStream`
* Refactor `_read_chunk_from`
* Convert main channel to 16-bit PCM for aioresonate compatibility
* Use separate host and advertise_host
Pass bind_ip for socket binding and publish_ip for mDNS advertising to
correctly handle scenarios where the server binds to all interfaces
(0.0.0.0) but advertises a specific IP address to clients.
* Bump `aioresonate`
* Add safety check to `TimedClientStream` to avoid memory leaks
* Update `MIN_BUFFER_DURATION` comment with findings
* Update music_assistant/providers/resonate/timed_client_stream.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Clean up `timed_client_stream` reference after playback
* Bump `aioresonate`
* Update call to the now async friendly `set_media_art`
* Update `playback_state` when joining a new group
* Update `group_members` based on `aioresonate` events
Instead of optimistically setting them.
* Move `MusicAssistantMediaStream` outside a method
* Run audio cleanup immediately
* Remove redundant type annotation
* Use `anext()`
* Remove unused async from `_cleanup_old_chunks`
* Simplify method name and clarify lock requirement
* Move `_generate` to class-level method
* Refactor `MusicAssistantMediaStream` parameters
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Remove explicit lvvmlite and numpy dependencies now that we are using debian.
Apple Music: Add remaining favourite parsing + custom music token config (#2609)
Add more error handling and logging to (different sample rate) crossfade actions
projects / music-assistant-server.git / log