5 Commits

Author SHA1 Message Date
fb2ae39b47 Add support for tracking and persisting snapshot IDs
Introduce `currentSnapshotID` to track the active snapshot and persist its state in a file. This change ensures the current snapshot ID is restored during initialization and maintained consistently across snapshot operations.
2025-05-10 13:12:17 +03:00
3efa753394 Quick bug fix 2025-05-10 01:31:33 +03:00
f7c1e461e6 Refactor snapshot methods to use active directory context
Updated `SaveSnapshot` and `RestoreSnapshot` methods to reference the active directory via `BlobStore`. Introduced `RestoreSnapshotToDir` for granular restore operations. Additionally, updated dependencies in `go.mod` to their latest versions for compatibility and maintenance.
2025-05-10 01:23:36 +03:00
b05058b5cd Add active directory management for snapshot operations
Introduced `GetActiveDir` and `CleanActiveDir` methods in the blob store to manage a dedicated directory for active snapshot operations. This ensures a clean working state before starting new operations and prevents conflicts. Updated related logic in snapshot creation and restoration to utilize the active directory.
2025-05-10 00:57:34 +03:00
9d04f43104 Updated package name 2025-05-08 11:32:21 +03:00
19 changed files with 528 additions and 60 deletions

View File

@ -6,7 +6,7 @@ download-third-party:
mv ./grpc/third_party/googleapis/grafeas ./grpc mv ./grpc/third_party/googleapis/grafeas ./grpc
rm -rf ./grpc/third_party rm -rf ./grpc/third_party
gen-proto-geolocation: gen-proto:
mkdir -p ./grpc mkdir -p ./grpc
@protoc -I ./grpc \ @protoc -I ./grpc \

310
README.md Normal file
View File

@ -0,0 +1,310 @@
# Agate
Agate is a Go library for creating, managing, and sharing snapshots of directories. It provides functionality for creating incremental snapshots, storing them efficiently, and sharing them over a network using gRPC.
## Installation
```bash
go get gitea.unprism.ru/KRBL/Agate
```
## Features
- Create snapshots of directories
- Incremental snapshots (only store changes)
- Restore snapshots
- List and manage snapshots
- Share snapshots over a network using gRPC
- Connect to remote snapshot repositories
## Basic Usage
### Creating a Snapshot Repository
To create a snapshot repository, you need to initialize the Agate library with the appropriate options:
```go
package main
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"gitea.unprism.ru/KRBL/Agate"
"gitea.unprism.ru/KRBL/Agate/stores"
)
func main() {
// Create directories for your repository
workDir := "/path/to/your/repository"
if err := os.MkdirAll(workDir, 0755); err != nil {
log.Fatalf("Failed to create work directory: %v", err)
}
// Initialize the default stores
metadataStore, blobStore, err := stores.InitDefaultStores(workDir)
if err != nil {
log.Fatalf("Failed to initialize stores: %v", err)
}
defer metadataStore.Close()
// Initialize Agate
agateOptions := agate.AgateOptions{
WorkDir: workDir,
MetadataStore: metadataStore,
BlobStore: blobStore,
}
ag, err := agate.New(agateOptions)
if err != nil {
log.Fatalf("Failed to initialize Agate: %v", err)
}
defer ag.Close()
// Create a snapshot
ctx := context.Background()
snapshotID, err := ag.SaveSnapshot(ctx, "My First Snapshot", "")
if err != nil {
log.Fatalf("Failed to create snapshot: %v", err)
}
fmt.Printf("Created snapshot with ID: %s\n", snapshotID)
// List snapshots
snapshots, err := ag.ListSnapshots(ctx)
if err != nil {
log.Fatalf("Failed to list snapshots: %v", err)
}
fmt.Printf("Found %d snapshots:\n", len(snapshots))
for _, s := range snapshots {
fmt.Printf(" - %s: %s (created at %s)\n", s.ID, s.Name, s.CreationTime.Format("2006-01-02 15:04:05"))
}
}
```
### Hosting a Snapshot Repository
To host a snapshot repository and make it available over the network, you can use the `StartServer` method:
```go
package main
import (
"context"
"log"
"os"
"os/signal"
"syscall"
"gitea.unprism.ru/KRBL/Agate"
"gitea.unprism.ru/KRBL/Agate/stores"
)
func main() {
// Create directories for your repository
workDir := "/path/to/your/repository"
if err := os.MkdirAll(workDir, 0755); err != nil {
log.Fatalf("Failed to create work directory: %v", err)
}
// Initialize the default stores
metadataStore, blobStore, err := stores.InitDefaultStores(workDir)
if err != nil {
log.Fatalf("Failed to initialize stores: %v", err)
}
defer metadataStore.Close()
// Initialize Agate
agateOptions := agate.AgateOptions{
WorkDir: workDir,
MetadataStore: metadataStore,
BlobStore: blobStore,
}
ag, err := agate.New(agateOptions)
if err != nil {
log.Fatalf("Failed to initialize Agate: %v", err)
}
defer ag.Close()
// Start the gRPC server
ctx := context.Background()
address := "0.0.0.0:50051" // Listen on all interfaces, port 50051
if err := ag.StartServer(ctx, address); err != nil {
log.Fatalf("Failed to start server: %v", err)
}
log.Printf("Server started on %s", address)
// Wait for termination signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
<-sigCh
log.Println("Shutting down...")
}
```
### Connecting to a Hosted Snapshot Repository
To connect to a hosted snapshot repository and retrieve snapshots:
```go
package main
import (
"context"
"fmt"
"log"
"os"
"gitea.unprism.ru/KRBL/Agate"
"gitea.unprism.ru/KRBL/Agate/stores"
)
func main() {
// Create directories for your local repository
workDir := "/path/to/your/local/repository"
if err := os.MkdirAll(workDir, 0755); err != nil {
log.Fatalf("Failed to create work directory: %v", err)
}
// Initialize the default stores
metadataStore, blobStore, err := stores.InitDefaultStores(workDir)
if err != nil {
log.Fatalf("Failed to initialize stores: %v", err)
}
defer metadataStore.Close()
// Initialize Agate
agateOptions := agate.AgateOptions{
WorkDir: workDir,
MetadataStore: metadataStore,
BlobStore: blobStore,
}
ag, err := agate.New(agateOptions)
if err != nil {
log.Fatalf("Failed to initialize Agate: %v", err)
}
defer ag.Close()
// Connect to a remote server
ctx := context.Background()
remoteAddress := "remote-server:50051"
// List snapshots from the remote server
snapshots, err := ag.GetRemoteSnapshotList(ctx, remoteAddress)
if err != nil {
log.Fatalf("Failed to list remote snapshots: %v", err)
}
fmt.Printf("Found %d remote snapshots:\n", len(snapshots))
for _, s := range snapshots {
fmt.Printf(" - %s: %s (created at %s)\n", s.ID, s.Name, s.CreationTime.Format("2006-01-02 15:04:05"))
}
// Download a specific snapshot
if len(snapshots) > 0 {
snapshotID := snapshots[0].ID
fmt.Printf("Downloading snapshot %s...\n", snapshotID)
// Download the snapshot (pass empty string as localParentID if this is the first download)
if err := ag.GetRemoteSnapshot(ctx, remoteAddress, snapshotID, ""); err != nil {
log.Fatalf("Failed to download snapshot: %v", err)
}
fmt.Printf("Successfully downloaded snapshot %s\n", snapshotID)
}
}
```
## Advanced Usage
### Creating Incremental Snapshots
You can create incremental snapshots by specifying a parent snapshot ID:
```go
// Create a first snapshot
snapshotID1, err := ag.SaveSnapshot(ctx, "First Snapshot", "")
if err != nil {
log.Fatalf("Failed to create first snapshot: %v", err)
}
// Make some changes to your files...
// Create a second snapshot with the first one as parent
snapshotID2, err := ag.SaveSnapshot(ctx, "Second Snapshot", snapshotID1)
if err != nil {
log.Fatalf("Failed to create second snapshot: %v", err)
}
```
### Restoring a Snapshot
To restore a snapshot:
```go
if err := ag.RestoreSnapshot(ctx, snapshotID); err != nil {
log.Fatalf("Failed to restore snapshot: %v", err)
}
```
### Getting Snapshot Details
To get detailed information about a snapshot:
```go
snapshot, err := ag.GetSnapshotDetails(ctx, snapshotID)
if err != nil {
log.Fatalf("Failed to get snapshot details: %v", err)
}
fmt.Printf("Snapshot: %s\n", snapshot.Name)
fmt.Printf("Created: %s\n", snapshot.CreationTime.Format("2006-01-02 15:04:05"))
fmt.Printf("Files: %d\n", len(snapshot.Files))
```
### Deleting a Snapshot
To delete a snapshot:
```go
if err := ag.DeleteSnapshot(ctx, snapshotID); err != nil {
log.Fatalf("Failed to delete snapshot: %v", err)
}
```
## API Reference
### Agate
The main entry point for the library.
- `New(options AgateOptions) (*Agate, error)` - Create a new Agate instance
- `SaveSnapshot(ctx context.Context, name string, parentID string) (string, error)` - Create a new snapshot
- `RestoreSnapshot(ctx context.Context, snapshotID string) error` - Restore a snapshot
- `ListSnapshots(ctx context.Context) ([]store.SnapshotInfo, error)` - List all snapshots
- `GetSnapshotDetails(ctx context.Context, snapshotID string) (*store.Snapshot, error)` - Get details of a snapshot
- `DeleteSnapshot(ctx context.Context, snapshotID string) error` - Delete a snapshot
- `StartServer(ctx context.Context, address string) error` - Start a gRPC server to share snapshots
- `ConnectRemote(address string) (*grpc.SnapshotClient, error)` - Connect to a remote server
- `GetRemoteSnapshotList(ctx context.Context, address string) ([]store.SnapshotInfo, error)` - List snapshots from a remote server
- `GetRemoteSnapshot(ctx context.Context, address string, snapshotID string, localParentID string) error` - Download a snapshot from a remote server
### AgateOptions
Configuration options for the Agate library.
- `WorkDir string` - Directory where snapshots will be stored and managed
- `OpenFunc func(dir string) error` - Called after a snapshot is restored
- `CloseFunc func() error` - Called before a snapshot is created or restored
- `MetadataStore store.MetadataStore` - Implementation of the metadata store
- `BlobStore store.BlobStore` - Implementation of the blob store
## License
[Add your license information here]

127
api.go
View File

@ -4,11 +4,11 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"gitea.unprism.ru/KRBL/Agate/grpc"
"os" "os"
"path/filepath" "path/filepath"
"unprism.ru/KRBL/agate/grpc"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// AgateOptions defines configuration options for the Agate library. // AgateOptions defines configuration options for the Agate library.
@ -36,10 +36,12 @@ type AgateOptions struct {
// Agate is the main entry point for the snapshot library. // Agate is the main entry point for the snapshot library.
type Agate struct { type Agate struct {
manager SnapshotManager manager SnapshotManager
options AgateOptions options AgateOptions
metadataDir string metadataDir string
blobsDir string blobsDir string
currentSnapshotID string
currentIDFile string
} }
// New initializes a new instance of the Agate library with the given options. // New initializes a new instance of the Agate library with the given options.
@ -90,16 +92,38 @@ func New(options AgateOptions) (*Agate, error) {
return nil, fmt.Errorf("failed to create snapshot manager: %w", err) return nil, fmt.Errorf("failed to create snapshot manager: %w", err)
} }
return &Agate{ // Create a file path for storing the current snapshot ID
manager: manager, currentIDFile := filepath.Join(options.WorkDir, "current_snapshot_id")
options: options,
metadataDir: metadataDir, agate := &Agate{
blobsDir: blobsDir, manager: manager,
}, nil options: options,
metadataDir: metadataDir,
blobsDir: blobsDir,
currentIDFile: currentIDFile,
}
// Load the current snapshot ID if it exists
if _, err := os.Stat(currentIDFile); err == nil {
data, err := os.ReadFile(currentIDFile)
if err == nil && len(data) > 0 {
agate.currentSnapshotID = string(data)
}
}
// Call OpenFunc if provided to initialize resources in the active directory
if options.OpenFunc != nil {
if err := options.OpenFunc(options.BlobStore.GetActiveDir()); err != nil {
return nil, fmt.Errorf("failed to open resources during initialization: %w", err)
}
}
return agate, nil
} }
// SaveSnapshot creates a new snapshot from the current state of the work directory. // SaveSnapshot creates a new snapshot from the current state of the active directory.
// If parentID is provided, it will be set as the parent of the new snapshot. // If parentID is provided, it will be set as the parent of the new snapshot.
// If parentID is empty, it will use the ID of the snapshot currently loaded in the active directory.
// Returns the ID of the created snapshot. // Returns the ID of the created snapshot.
func (a *Agate) SaveSnapshot(ctx context.Context, name string, parentID string) (string, error) { func (a *Agate) SaveSnapshot(ctx context.Context, name string, parentID string) (string, error) {
// Call CloseFunc if provided // Call CloseFunc if provided
@ -109,15 +133,26 @@ func (a *Agate) SaveSnapshot(ctx context.Context, name string, parentID string)
} }
} }
// If parentID is not provided, use the current snapshot ID
effectiveParentID := parentID
// Create the snapshot // Create the snapshot
snapshot, err := a.manager.CreateSnapshot(ctx, a.options.WorkDir, name, parentID) snapshot, err := a.manager.CreateSnapshot(ctx, a.options.BlobStore.GetActiveDir(), name, effectiveParentID)
if err != nil { if err != nil {
return "", fmt.Errorf("failed to create snapshot: %w", err) return "", fmt.Errorf("failed to create snapshot: %w", err)
} }
// Update the current snapshot ID to the newly created snapshot
a.currentSnapshotID = snapshot.ID
// Save the current snapshot ID to a file
if err := a.saveCurrentSnapshotID(); err != nil {
return "", fmt.Errorf("failed to save current snapshot ID: %w", err)
}
// Call OpenFunc if provided // Call OpenFunc if provided
if a.options.OpenFunc != nil { if a.options.OpenFunc != nil {
if err := a.options.OpenFunc(a.options.WorkDir); err != nil { if err := a.options.OpenFunc(a.options.BlobStore.GetActiveDir()); err != nil {
return "", fmt.Errorf("failed to open resources after snapshot: %w", err) return "", fmt.Errorf("failed to open resources after snapshot: %w", err)
} }
} }
@ -125,7 +160,7 @@ func (a *Agate) SaveSnapshot(ctx context.Context, name string, parentID string)
return snapshot.ID, nil return snapshot.ID, nil
} }
// RestoreSnapshot extracts a snapshot to the work directory. // RestoreSnapshot extracts a snapshot to the active directory.
func (a *Agate) RestoreSnapshot(ctx context.Context, snapshotID string) error { func (a *Agate) RestoreSnapshot(ctx context.Context, snapshotID string) error {
// Call CloseFunc if provided // Call CloseFunc if provided
if a.options.CloseFunc != nil { if a.options.CloseFunc != nil {
@ -135,13 +170,55 @@ func (a *Agate) RestoreSnapshot(ctx context.Context, snapshotID string) error {
} }
// Extract the snapshot // Extract the snapshot
if err := a.manager.ExtractSnapshot(ctx, snapshotID, a.options.WorkDir); err != nil { if err := a.manager.ExtractSnapshot(ctx, snapshotID, a.options.BlobStore.GetActiveDir()); err != nil {
return fmt.Errorf("failed to extract snapshot: %w", err) return fmt.Errorf("failed to extract snapshot: %w", err)
} }
// Save the ID of the snapshot that was restored
a.currentSnapshotID = snapshotID
// Save the current snapshot ID to a file
if err := a.saveCurrentSnapshotID(); err != nil {
return fmt.Errorf("failed to save current snapshot ID: %w", err)
}
// Call OpenFunc if provided // Call OpenFunc if provided
if a.options.OpenFunc != nil { if a.options.OpenFunc != nil {
if err := a.options.OpenFunc(a.options.WorkDir); err != nil { if err := a.options.OpenFunc(a.options.BlobStore.GetActiveDir()); err != nil {
return fmt.Errorf("failed to open resources after restore: %w", err)
}
}
return nil
}
// RestoreSnapshot extracts a snapshot to the directory.
func (a *Agate) RestoreSnapshotToDir(ctx context.Context, snapshotID string, dir string) error {
// Call CloseFunc if provided
if a.options.CloseFunc != nil {
if err := a.options.CloseFunc(); err != nil {
return fmt.Errorf("failed to close resources before restore: %w", err)
}
}
// Extract the snapshot
if err := a.manager.ExtractSnapshot(ctx, snapshotID, dir); err != nil {
return fmt.Errorf("failed to extract snapshot: %w", err)
}
// If restoring to the active directory, save the snapshot ID
if dir == a.options.BlobStore.GetActiveDir() {
a.currentSnapshotID = snapshotID
// Save the current snapshot ID to a file
if err := a.saveCurrentSnapshotID(); err != nil {
return fmt.Errorf("failed to save current snapshot ID: %w", err)
}
}
// Call OpenFunc if provided
if a.options.OpenFunc != nil {
if err := a.options.OpenFunc(dir); err != nil {
return fmt.Errorf("failed to open resources after restore: %w", err) return fmt.Errorf("failed to open resources after restore: %w", err)
} }
} }
@ -164,6 +241,20 @@ func (a *Agate) DeleteSnapshot(ctx context.Context, snapshotID string) error {
return a.manager.DeleteSnapshot(ctx, snapshotID) return a.manager.DeleteSnapshot(ctx, snapshotID)
} }
// saveCurrentSnapshotID saves the current snapshot ID to a file in the WorkDir
func (a *Agate) saveCurrentSnapshotID() error {
if a.currentSnapshotID == "" {
// If there's no current snapshot ID, remove the file if it exists
if _, err := os.Stat(a.currentIDFile); err == nil {
return os.Remove(a.currentIDFile)
}
return nil
}
// Write the current snapshot ID to the file
return os.WriteFile(a.currentIDFile, []byte(a.currentSnapshotID), 0644)
}
// Close releases all resources used by the Agate instance. // Close releases all resources used by the Agate instance.
func (a *Agate) Close() error { func (a *Agate) Close() error {
// Currently, we don't have a way to close the manager directly // Currently, we don't have a way to close the manager directly

View File

@ -7,8 +7,8 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"unprism.ru/KRBL/agate" "gitea.unprism.ru/KRBL/Agate"
"unprism.ru/KRBL/agate/stores" "gitea.unprism.ru/KRBL/Agate/stores"
) )
func main() { func main() {

12
go.mod
View File

@ -1,4 +1,4 @@
module unprism.ru/KRBL/agate module gitea.unprism.ru/KRBL/Agate
go 1.24.0 go 1.24.0
@ -6,14 +6,14 @@ require (
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3
github.com/mattn/go-sqlite3 v1.14.28 github.com/mattn/go-sqlite3 v1.14.28
google.golang.org/genproto/googleapis/api v0.0.0-20250422160041-2d3770c4ea7f google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2
google.golang.org/grpc v1.72.0 google.golang.org/grpc v1.72.0
google.golang.org/protobuf v1.36.6 google.golang.org/protobuf v1.36.6
) )
require ( require (
golang.org/x/net v0.39.0 // indirect golang.org/x/net v0.40.0 // indirect
golang.org/x/sys v0.32.0 // indirect golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.24.0 // indirect golang.org/x/text v0.25.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250422160041-2d3770c4ea7f // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20250505200425-f936aa4a68b2 // indirect
) )

10
go.sum
View File

@ -26,14 +26,24 @@ go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE= go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY= golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E= golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20= golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0= golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU= golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
google.golang.org/genproto/googleapis/api v0.0.0-20250422160041-2d3770c4ea7f h1:tjZsroqekhC63+WMqzmWyW5Twj/ZfR5HAlpd5YQ1Vs0= google.golang.org/genproto/googleapis/api v0.0.0-20250422160041-2d3770c4ea7f h1:tjZsroqekhC63+WMqzmWyW5Twj/ZfR5HAlpd5YQ1Vs0=
google.golang.org/genproto/googleapis/api v0.0.0-20250422160041-2d3770c4ea7f/go.mod h1:Cd8IzgPo5Akum2c9R6FsXNaZbH3Jpa2gpHlW89FqlyQ= google.golang.org/genproto/googleapis/api v0.0.0-20250422160041-2d3770c4ea7f/go.mod h1:Cd8IzgPo5Akum2c9R6FsXNaZbH3Jpa2gpHlW89FqlyQ=
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2 h1:vPV0tzlsK6EzEDHNNH5sa7Hs9bd7iXR7B1tSiPepkV0=
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2/go.mod h1:pKLAc5OolXC3ViWGI62vvC0n10CpwAtRcTNCFwTKBEw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250422160041-2d3770c4ea7f h1:N/PrbTw4kdkqNRzVfWPrBekzLuarFREcbFOiOLkXon4= google.golang.org/genproto/googleapis/rpc v0.0.0-20250422160041-2d3770c4ea7f h1:N/PrbTw4kdkqNRzVfWPrBekzLuarFREcbFOiOLkXon4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250422160041-2d3770c4ea7f/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A= google.golang.org/genproto/googleapis/rpc v0.0.0-20250422160041-2d3770c4ea7f/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250505200425-f936aa4a68b2 h1:IqsN8hx+lWLqlN+Sc3DoMy/watjofWiU8sRFgQ8fhKM=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250505200425-f936aa4a68b2/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.72.0 h1:S7UkcVa60b5AAQTaO6ZKamFp1zMZSU0fGDK2WZLbBnM= google.golang.org/grpc v1.72.0 h1:S7UkcVa60b5AAQTaO6ZKamFp1zMZSU0fGDK2WZLbBnM=
google.golang.org/grpc v1.72.0/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM= google.golang.org/grpc v1.72.0/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY= google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=

View File

@ -10,7 +10,7 @@ import (
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// SnapshotClient implements the client for connecting to a remote snapshot server // SnapshotClient implements the client for connecting to a remote snapshot server

View File

@ -9,8 +9,8 @@ import (
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/protobuf/types/known/timestamppb" "google.golang.org/protobuf/types/known/timestamppb"
"unprism.ru/KRBL/agate/interfaces" "gitea.unprism.ru/KRBL/Agate/interfaces"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// SnapshotServer implements the gRPC server for snapshots // SnapshotServer implements the gRPC server for snapshots

View File

@ -476,7 +476,7 @@ const file_snapshot_proto_rawDesc = "" +
"\x0fSnapshotService\x12k\n" + "\x0fSnapshotService\x12k\n" +
"\rListSnapshots\x12 .agate.grpc.ListSnapshotsRequest\x1a!.agate.grpc.ListSnapshotsResponse\"\x15\x82\xd3\xe4\x93\x02\x0f\x12\r/v1/snapshots\x12}\n" + "\rListSnapshots\x12 .agate.grpc.ListSnapshotsRequest\x1a!.agate.grpc.ListSnapshotsResponse\"\x15\x82\xd3\xe4\x93\x02\x0f\x12\r/v1/snapshots\x12}\n" +
"\x12GetSnapshotDetails\x12%.agate.grpc.GetSnapshotDetailsRequest\x1a\x1b.agate.grpc.SnapshotDetails\"#\x82\xd3\xe4\x93\x02\x1d\x12\x1b/v1/snapshots/{snapshot_id}\x12\x8a\x01\n" + "\x12GetSnapshotDetails\x12%.agate.grpc.GetSnapshotDetailsRequest\x1a\x1b.agate.grpc.SnapshotDetails\"#\x82\xd3\xe4\x93\x02\x1d\x12\x1b/v1/snapshots/{snapshot_id}\x12\x8a\x01\n" +
"\fDownloadFile\x12\x1f.agate.grpc.DownloadFileRequest\x1a .agate.grpc.DownloadFileResponse\"5\x82\xd3\xe4\x93\x02/\x12-/v1/snapshots/{snapshot_id}/files/{file_path}0\x01B\x1cZ\x1aunprism.ru/KRBL/agate/grpcb\x06proto3" "\fDownloadFile\x12\x1f.agate.grpc.DownloadFileRequest\x1a .agate.grpc.DownloadFileResponse\"5\x82\xd3\xe4\x93\x02/\x12-/v1/snapshots/{snapshot_id}/files/{file_path}0\x01B\"Z gitea.unprism.ru/KRBL/Agate/grpcb\x06proto3"
var ( var (
file_snapshot_proto_rawDescOnce sync.Once file_snapshot_proto_rawDescOnce sync.Once

View File

@ -5,7 +5,7 @@ package agate.grpc;
import "google/protobuf/timestamp.proto"; import "google/protobuf/timestamp.proto";
import "google/api/annotations.proto"; // Добавлено для HTTP mapping import "google/api/annotations.proto"; // Добавлено для HTTP mapping
option go_package = "unprism.ru/KRBL/agate/grpc"; option go_package = "gitea.unprism.ru/KRBL/Agate/grpc";
// Сервис для управления снапшотами // Сервис для управления снапшотами
service SnapshotService { service SnapshotService {

View File

@ -4,7 +4,7 @@ import (
"context" "context"
"io" "io"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// SnapshotManager defines the interface that the server needs to interact with snapshots // SnapshotManager defines the interface that the server needs to interact with snapshots

View File

@ -13,9 +13,9 @@ import (
"github.com/google/uuid" "github.com/google/uuid"
"unprism.ru/KRBL/agate/archive" "gitea.unprism.ru/KRBL/Agate/archive"
"unprism.ru/KRBL/agate/hash" "gitea.unprism.ru/KRBL/Agate/hash"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
type SnapshotManagerData struct { type SnapshotManagerData struct {
@ -59,12 +59,20 @@ func (data *SnapshotManagerData) CreateSnapshot(ctx context.Context, sourceDir s
// Generate a unique ID for the snapshot // Generate a unique ID for the snapshot
snapshotID := uuid.New().String() snapshotID := uuid.New().String()
// Create a temporary file for the archive // Clean the active directory to avoid conflicts
tempFile, err := os.CreateTemp("", "agate-snapshot-*.zip") if err := data.blobStore.CleanActiveDir(ctx); err != nil {
if err != nil { return nil, fmt.Errorf("failed to clean active directory: %w", err)
return nil, fmt.Errorf("failed to create temporary file: %w", err) }
// Get the active directory for operations
activeDir := data.blobStore.GetActiveDir()
// Create a temporary file for the archive in the active directory
tempFilePath := filepath.Join(activeDir, "temp-"+snapshotID+".zip")
tempFile, err := os.Create(tempFilePath)
if err != nil {
return nil, fmt.Errorf("failed to create temporary file in active directory: %w", err)
} }
tempFilePath := tempFile.Name()
tempFile.Close() // Close it as CreateArchive will reopen it tempFile.Close() // Close it as CreateArchive will reopen it
defer os.Remove(tempFilePath) // Clean up temp file after we're done defer os.Remove(tempFilePath) // Clean up temp file after we're done
@ -253,8 +261,15 @@ func (data *SnapshotManagerData) ExtractSnapshot(ctx context.Context, snapshotID
if snapshotID == "" { if snapshotID == "" {
return errors.New("snapshot ID cannot be empty") return errors.New("snapshot ID cannot be empty")
} }
// If no specific path is provided, use the active directory
if path == "" { if path == "" {
return errors.New("target path cannot be empty") // Clean the active directory to avoid conflicts
if err := data.blobStore.CleanActiveDir(ctx); err != nil {
return fmt.Errorf("failed to clean active directory: %w", err)
}
path = filepath.Join(data.blobStore.GetActiveDir(), snapshotID)
} }
// First check if the snapshot exists // First check if the snapshot exists

View File

@ -10,9 +10,9 @@ import (
stdgrpc "google.golang.org/grpc" stdgrpc "google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
agateGrpc "unprism.ru/KRBL/agate/grpc" agateGrpc "gitea.unprism.ru/KRBL/Agate/grpc"
"unprism.ru/KRBL/agate/interfaces" "gitea.unprism.ru/KRBL/Agate/interfaces"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// Client represents a client for connecting to a remote snapshot server // Client represents a client for connecting to a remote snapshot server

View File

@ -9,9 +9,9 @@ import (
stdgrpc "google.golang.org/grpc" stdgrpc "google.golang.org/grpc"
"google.golang.org/protobuf/types/known/timestamppb" "google.golang.org/protobuf/types/known/timestamppb"
agateGrpc "unprism.ru/KRBL/agate/grpc" agateGrpc "gitea.unprism.ru/KRBL/Agate/grpc"
"unprism.ru/KRBL/agate/interfaces" "gitea.unprism.ru/KRBL/Agate/interfaces"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
) )
// Server implements the gRPC server for snapshots // Server implements the gRPC server for snapshots

View File

@ -2,8 +2,8 @@ package agate
import ( import (
"context" "context"
"gitea.unprism.ru/KRBL/Agate/store"
"io" "io"
"unprism.ru/KRBL/agate/store"
) )
// SnapshotManager is an interface that defines operations for managing and interacting with snapshots. // SnapshotManager is an interface that defines operations for managing and interacting with snapshots.

View File

@ -3,27 +3,38 @@ package filesystem
import ( import (
"context" "context"
"fmt" "fmt"
"gitea.unprism.ru/KRBL/Agate"
"gitea.unprism.ru/KRBL/Agate/store"
"io" "io"
"os" "os"
"path/filepath" "path/filepath"
"unprism.ru/KRBL/agate"
"unprism.ru/KRBL/agate/store"
) )
const blobExtension = ".zip" const blobExtension = ".zip"
// fileSystemStore реализует интерфейс store.BlobStore с использованием локальной файловой системы. // fileSystemStore реализует интерфейс store.BlobStore с использованием локальной файловой системы.
type fileSystemStore struct { type fileSystemStore struct {
baseDir string // Директория для хранения блобов (архивов) baseDir string // Директория для хранения блобов (архивов)
activeDir string // Директория для активных операций (создание и восстановление)
} }
// NewFileSystemStore создает новое хранилище блобов в указанной директории. // NewFileSystemStore создает новое хранилище блобов в указанной директории.
func NewFileSystemStore(baseDir string) (store.BlobStore, error) { func NewFileSystemStore(baseDir string) (store.BlobStore, error) {
// Убедимся, что директория существует // Убедимся, что базовая директория существует
if err := os.MkdirAll(baseDir, 0755); err != nil { if err := os.MkdirAll(baseDir, 0755); err != nil {
return nil, fmt.Errorf("failed to create base directory %s for filesystem blob store: %w", baseDir, err) return nil, fmt.Errorf("failed to create base directory %s for filesystem blob store: %w", baseDir, err)
} }
return &fileSystemStore{baseDir: baseDir}, nil
// Создаем директорию для активных операций внутри базовой директории
activeDir := filepath.Join(baseDir, "active")
if err := os.MkdirAll(activeDir, 0755); err != nil {
return nil, fmt.Errorf("failed to create active directory %s for filesystem blob store: %w", activeDir, err)
}
return &fileSystemStore{
baseDir: baseDir,
activeDir: activeDir,
}, nil
} }
// getBlobPath формирует полный путь к файлу блоба. // getBlobPath формирует полный путь к файлу блоба.
@ -106,3 +117,27 @@ func (fs *fileSystemStore) GetBlobPath(ctx context.Context, snapshotID string) (
// Файл существует, возвращаем путь // Файл существует, возвращаем путь
return blobPath, nil return blobPath, nil
} }
// GetActiveDir возвращает путь к директории для активных операций.
func (fs *fileSystemStore) GetActiveDir() string {
return fs.activeDir
}
// CleanActiveDir очищает директорию для активных операций.
// Это полезно перед началом новых операций, чтобы избежать конфликтов.
func (fs *fileSystemStore) CleanActiveDir(ctx context.Context) error {
// Удаляем все файлы в активной директории, но сохраняем саму директорию
entries, err := os.ReadDir(fs.activeDir)
if err != nil {
return fmt.Errorf("failed to read active directory: %w", err)
}
for _, entry := range entries {
path := filepath.Join(fs.activeDir, entry.Name())
if err := os.RemoveAll(path); err != nil {
return fmt.Errorf("failed to remove %s from active directory: %w", path, err)
}
}
return nil
}

View File

@ -6,12 +6,12 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"gitea.unprism.ru/KRBL/Agate"
"gitea.unprism.ru/KRBL/Agate/store"
_ "github.com/mattn/go-sqlite3" _ "github.com/mattn/go-sqlite3"
"os" "os"
"path/filepath" "path/filepath"
"time" "time"
"unprism.ru/KRBL/agate"
"unprism.ru/KRBL/agate/store"
) )
const ( const (

View File

@ -71,4 +71,11 @@ type BlobStore interface {
// Это может быть полезно для функций пакета archive, которые работают с путями. // Это может быть полезно для функций пакета archive, которые работают с путями.
// Возвращает agate.ErrNotFound, если блоб не найден. // Возвращает agate.ErrNotFound, если блоб не найден.
GetBlobPath(ctx context.Context, snapshotID string) (string, error) GetBlobPath(ctx context.Context, snapshotID string) (string, error)
// GetActiveDir возвращает путь к директории для активных операций (создание и восстановление).
GetActiveDir() string
// CleanActiveDir очищает директорию для активных операций.
// Это полезно перед началом новых операций, чтобы избежать конфликтов.
CleanActiveDir(ctx context.Context) error
} }

View File

@ -4,9 +4,9 @@ import (
"fmt" "fmt"
"path/filepath" "path/filepath"
"unprism.ru/KRBL/agate/store" "gitea.unprism.ru/KRBL/Agate/store"
"unprism.ru/KRBL/agate/store/filesystem" "gitea.unprism.ru/KRBL/Agate/store/filesystem"
"unprism.ru/KRBL/agate/store/sqlite" "gitea.unprism.ru/KRBL/Agate/store/sqlite"
) )
// NewDefaultMetadataStore creates a new SQLite-based metadata store. // NewDefaultMetadataStore creates a new SQLite-based metadata store.